Every way to order memory between threads
What x86-TSO already promises, full fences (mfence, dmb ish, fence rw,rw), lighter one-way fences, release stores and acquire loads, locked instructions, and the C++ memory orders that pick among them.
Ask which two accesses must stay in order, then use the lightest tool that keeps them.
- Publishing data behind a flag (write data, then the flag; read
the flag, then the data) → a release store and an acquire load. C++
memory_order_release/acquire. On x86-64 these are plainmovs; on ARM64stlr/ldar; on RISC-Vfence rw, wbefore the store andfence r, rwafter the load. - A store followed by a load of a different variable that must not
pass it (Dekker-style flags,
seq_cst) → a full barrier:mfenceor any locked instruction /xchgon x86-64,dmb ishon ARM64,fence rw, rwon RISC-V. This is the one reordering x86-TSO allows. - Only stores against stores, or loads against loads → the
one-way fences
dmb ishst/dmb ishld,fence w, w/fence r, r. - You only need the compiler not to reorder (a signal handler on the same thread) → a compiler barrier, no instruction at all.
None of these flush caches: coherence already keeps caches agreeing. Barriers only stop this core from letting later accesses go ahead.
What the hardware already promises
Formal. At any moment, each cache line is either
- writable by exactly one core (MESI state Modified or Exclusive in that core's cache, Invalid everywhere else), or
- readable by any number of cores (Shared or Invalid everywhere),
and a read returns the value of the most recent write to that location in the single order of writes to it. A core that wants to write a line it holds Shared must first invalidate every other copy.
In plain English. Many cores may read a line at once, but only one may write it, and while it writes, nobody else has a copy. Every core therefore agrees on the history of each memory location.
Example. Cores 0 and 1 both read x: the line is Shared in both caches. Core 1 writes x: it sends an invalidation, core 0's copy becomes Invalid and core 1's becomes Modified. Core 0 reads x again: it misses (a coherence miss), core 1 supplies the new value, and both copies become Shared.
Why this rule exists. It is what lets private per-core caches act like one memory for each location. It also predicts the cost: every write to a line that other cores hold moves the line, even when the cores are using different bytes of it.
Section
Cache coherence and MESI
One location is never seen two ways at once; ordering between locations is a separate question.
Formal. Under x86-TSO, for memory operations of one core to different addresses, as observed by other cores:
| earlier → later | may be reordered? |
|---|---|
| load → load | no |
| load → store | no |
| store → store | no |
| store → load | yes: the later load may complete before the earlier store becomes visible |
All stores become visible to all other cores in one total order, a core
reads its own stores early through store-to-load forwarding, and a
locked instruction or mfence forbids the store → load case across it.
In plain English. On x86-64, other cores see a core's loads in order and its stores in order. The one exception: a store can sit in the store buffer while a later load to a different address goes ahead.
Example. The store-buffering litmus test, x = y = 0 at the start:
thread 1 runs mov [x], 1 then mov eax, [y]; thread 2 runs
mov [y], 1 then mov eax, [x]. Under sequential consistency at least
one thread must read 1. Under x86-TSO both may read 0, because each
store can still be in its core's store buffer when the other core's load
runs. An mfence between the store and the load in each thread forbids
that outcome.
Why this rule exists. It is the whole difference between x86-64 and sequential consistency, so it tells exactly which x86-64 code needs a barrier: code that stores one shared variable and then loads a different one and relies on the order.
Section
Store buffers and x86-TSO
x86-64 lets only a later load pass an earlier store.
Almost any reordering is allowed unless a barrier or dependency forbids it.
Full and one-way barriers
Example
mfence (x86-64), dmb ish (ARM64), fence rw, rw (RISC-V): nothing before it may be passed by anything after it.Term
lock prefix
Every locked instruction, and xchg with memory, is also a full barrier on x86-64.
One-way ordering on the access itself
Formal. If thread A performs a release store to a location f, and thread B performs an acquire load of f that reads the value A stored, then every memory write A performed before its release store (in A's program order) is visible to every read B performs after its acquire load. A release store keeps all earlier loads and stores before it; an acquire load keeps all later loads and stores after it.
| x86-64 | ARM64 | RISC-V | |
|---|---|---|---|
| release store | plain mov |
stlr |
fence rw,w then store |
| acquire load | plain mov |
ldar (or ldapr) |
load then fence r,rw |
In plain English. Write the data first, then set the flag with a release store. Read the flag with an acquire load, and if it is set, the data is guaranteed to be there. Each side only keeps order in one direction, which is cheaper than a full barrier.
Example. A: data = 42; ready.store(1, std::memory_order_release);
B: while (!ready.load(std::memory_order_acquire)) {} use(data); — B
always sees 42. With both operations relaxed, ARM64 may let B see
ready == 1 and still read the old data; on x86-64 the hardware keeps
this order anyway, but the compiler may not, so the C++ orders are still
required.
Why this rule exists. Publishing data from one thread to another is the most common pattern in concurrent code, and acquire/release is the cheapest ordering that makes it correct on every ISA.
Section
Acquire, release and C++ atomics
How each memory_order compiles on the three ISAs.