From Newsgroup: comp.arch
anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
Paul Clayton <paaronclayton@gmail.com> writes:
I seem to recall reading that x86's LOCK instructions take
hundreds of cycles. While some of this is probably from stronger
memory ordering guarantees, I get the impression that the
operation itself is not aggressively optimized.
I have revised the benchmarks as follows: I have added a test of a
memory barrier, which is implemented in GNU C as
__atomic_thread_fence(__ATOMIC_SEQ_CST);
The barriers separate loads.
I have increased the loop count by a factor of 10, because I did not
subtract the startup overhead of Gforth; as a result, the startup
overhead is reduced from 3.3 cycles per execution of the relevant word
to 0.33 cycles.
I have also inserted 64 bytes between the variables, so that they are
in different cache lines. This should not make a difference, because
all accesses are in the same thread (i.e., no cache-ping-pong from
possible false sharing), but just in case.
What I did not do is to use several threads. The idea here is that
programmers will take measures that ensure that contention is rare,
but you still need to use atomic instructions and barriers to ensure correctness. Ideally in this case the atomic instructions and
barriers have no extra cost, but in reality, they do have extra cost.
If you are interested in seeing data for the contended case, look at
the cache ping-pong benchmarks, e.g., on chipsandcheese. There is one
danger in my approach: Hardware could have a special optimization for
memory that is not shared between threads at all, and run slower if
the memory is shared, but not contended; I have never read about such
a mechanism, and I'll leave checking the performance with multiple non-contending threads for another day.
The source code now is:
variable x 1 x !
64 allot \ make sure the variables are in different cache lines
variable y -1 y !
: bench-!@
1 50_000_000 0 do x !@ y !@ loop drop ;
: bench-atomic!@
1 50_000_000 0 do x atomic!@ y atomic!@ loop drop ;
: bench-+!@
1 50_000_000 0 do x +!@ y +!@ loop drop ;
: bench-atomic+!@
1 50_000_000 0 do x atomic+!@ y atomic+!@ loop drop ;
: bench-nobarrier
50_000_000 0 do x @ y @ 2drop loop ;
: bench-barrier
50_000_000 0 do x @ barrier y @ barrier 2drop loop ;
The results are:
Ryzen 8700G (Zen4):
!@ +!@ barr
2.4 2.4 1.8 no atomic/no barrier
9.2 8.3 7.1 atomic/barrier
Ryzen 3900X (Zen2; in contrast to the 8700G with 1 CCX, the 3900X has
4 CCXs that may need coordination):
!@ +!@ barr
2.9 4.5 2.2 no atomic/no barrier
19.1 19.0 17.5 atomic/barrier
Given that the cycles here are far below the cycles reported for
Inter-CCX cache ping-pong, I guess that there is no inter-CCX
communication (at least no bidirectional one) in this benchmark.
On to Intel:
Core i3-1315U P-core (Golden Cove):
!@ +!@ barr
1.9 1.9 1.5 no atomic/no barrier
19.4 20.9 27.9 atomic/barrier
Core i3-1315U E-core (Gracemont):
!@ +!@ barr
2.7 2.2 2.2 no atomic/no barrier
20.6 20.4 20.0 atomic/barrier
On to Apple Silicon (weak memory ordering by default):
Apple M1 P-core (Firestorm):
!@ +!@ barr
3.6 3.6 3.5 no atomic/no barrier
31.9 31.5 3.6 atomic/barrier
Apple M1 E-core (Icestorm):
!@ +!@ barr
3.4 3.4 3.4 no atomic/no barrier
31.4 32.9 6.9 atomic/barrier
On to ARM (weak memory ordering):
RK3588 big (Cortex-A76):
!@ +!@ barr
3.3 3.6 3.3 no atomic/no barrier
20.3 20.4 13.2 atomic/barrier
RK3588 little (Cortex-A55):
!@ +!@ barr
7.2 9.2 7.2 no atomic/no barrier
68.1 57.1 16.2 atomic/barrier
I find the cheapness of the barrier on the M1 surprising. I would
have expected that barriers are more expensive on hardware where the architecture allows more reordering and the hardware makes use of that
license (and I think that the M1 does make use of it).
OTOH, the atomic stuff is more expensive on the Apple M1 and the ARM
cores than on the Intel and AMD cores (note that the cycle times of
the Intel and AMD cores used here is quite a bit shorter than for
Apple and ARM cores, except for Gracemont compared to Firestorm; but
for Firestorm the number of cycles executed is higher, so Gracemont
still takes less time.
In conclusion, as long as we have no contention, atomic accesses and
barriers do not cost hundreds of cycles, but they do cost enough extra
(except the barrier on Firestorm, at least in the present benchmark)
that one does not want to use them across the board, only when
accessing memory that another thread accesses, too. At least in this
sample of cores, the atomic instructions are faster on Intel and AMD
cores than on Apple and ARM cores; for the barrier, the costs are
usually not higher and sometimes significantly cheaper than for the
atomic instructions.
- anton
On a Ryzen 8700G (Zen4) each execution of a !@ (exchange) or +!@ >(fetch-and-add) costs the following numbers of cycles (including
overhead):
!@ +!@
7.5 7.3 not atomic
14.2 13.2 atomic
On a Xeon E-2388G (Rocket Lake):
!@ +!@
8.5 7.1 not atomic
25.8 26.6 atomic
- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <
c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>
--- Synchronet 3.22a-Linux NewsLink 1.2