With FMA instructions V* executes fewer instructions (6 per iteration
instead of 7), but more cycles (4 per iteration instead of 3). My explanation is that the FMA instruction has 4 cycles of latency on
Zen4, while fadd has 3 cycles of latency.
I discussed summing up an array with pairwise addition earlier <2025Jul16.132504@mips.complang.tuwien.ac.at>. The benefits compared
to the straightforward algorithm are:
* Better performance, because the straightforward algorithm is limited
by the dependence chains of the F+ computations.
* Lower rounding errors in the typical case.
The dot-product of two vectors is a primitive in linear algebra, and
also has such a chain of F+ if implemented natively. Implementing the
dot product with pairwise addition should have similar benefits.
Gforth already has a word for the dot product:
|'v*' ( f-addr1 nstride1 f-addr2 nstride2 ucount - r ) gforth-0.5 "v-star"
| dot-product: r=v1*v2. The first element of v1 is at f_addr1, the
|next at f_addr1+nstride1 and so on (similar for v2). Both vectors have |ucount elements.
This word is implemented as C function, which allows us to compare the performance of the various Forth systems to C.
I have implemented variants of v* in Forth, a straightforward one and
one with pairwise addition. The code can be found in
https://www.complang.tuwien.ac.at/forth/programs/pairwise-fdot.4th
Note that if your Forth system only supports an 8-deep FP stack, you
must not call PAIRWISE-V* unless the FP stack is empty.
I have measured the performance for 10000 dot products of vectors of
50000 elements (so the straightforward algorithm has a minimum
run-time of 0.5G times the latency of F+.
Here are the results (measured on Zen4 (Ryzen 8700G)):
gforth-fast gforth-fast-v3 lxf sf64 vfx64
cycles:u
1_543_071_551 2_044_522_195 v*
6_095_056_325 6_544_418_067 3_221_307_358 4_579_918_381 3_425_243_416 st
4_350_367_901 4_371_526_269 10_527_316_831 10_630_319_712 1_203_870_244 pw
instructions:u
3_618_080_286 3_117_767_051 v* 11_618_280_866 11_617_967_567 9_516_121_645 17_010_999_668 6_074_024_083 st 10_164_410_950 10_164_097_601 14_265_222_850 16_962_950_438 3_823_084_163 pw
The Forth versions are:
gforth-fast current development version
gforth-fast-v3 current development version compiled with -march=x86-64-v3
this makes use of fma instructions
lxf 1.6-982-823
sf64 SwiftForth 4.0.0-RC89
vfx64 5.43
The versions are:
tag word Explanation
v* v* Gforth primitive implemented in C
st naive-v* Straightforward implementation
pw pairwise-v*
VFX64 shows that pairwise addition can be faster than even the C implementation of the straightforward algorithm (v* in gforth-fast).
VFX also executes slightly fewer instructions than V* on Gforth-fast:
V* has loop overhead, whereas PAIRWISE-V* uses base steps in the
recursion for vectors with up to 64 elements, resulting in less
overhead.
With FMA instructions V* executes fewer instructions (6 per iteration
instead of 7), but more cycles (4 per iteration instead of 3). My explanation is that the FMA instruction has 4 cycles of latency on
Zen4, while fadd has 3 cycles of latency. Interestingly, in
gforth-fast-v3 NAIVE-V* takes the same number of instructions and one
cycle per iteration more than in gforth-fast, but I have not looked at
why that is.
In gforth-fast the PAIRWISE-V* is quite a bit faster than NAIVE-V*.
It also takes a little fewer instructions, probably because of the
unfolding. But V* is still a lot faster, so our code generation
should be improved.
I have not looked at why lxf and sf64 perform word on PAIRWISE-V* than
on NAIVE-V*.
- anton
I tested this on lxf64 running on a ryzen 9950x
I got
cycles instructions
pairwise 789,294,432 3,628,952,994
naive 2,510,536,091 6,512,423,204
First run with naive I got 35,476,814,233 cycles and 54,012,383,854 instr.
I saw that you checked for F*+ . I have that as a library call, obviously >clang does not use a fma instruction for it but instead compiles a call
to a libray function.
I would guess that lxf is limited by the use of the x87 fpus small stack
But I need to confirm that
I discussed summing up an array with pairwise addition earlier ><2025Jul16.132504@mips.complang.tuwien.ac.at>. The benefits compared
to the straightforward algorithm are:
* Better performance, because the straightforward algorithm is limited
by the dependence chains of the F+ computations.
* Lower rounding errors in the typical case.
The dot-product of two vectors is a primitive in linear algebra, and
also has such a chain of F+ if implemented natively. Implementing the
dot product with pairwise addition should have similar benefits.
Gforth already has a word for the dot product:
|'v*' ( f-addr1 nstride1 f-addr2 nstride2 ucount - r ) gforth-0.5 "v-star"
| dot-product: r=v1*v2. The first element of v1 is at f_addr1, the
|next at f_addr1+nstride1 and so on (similar for v2). Both vectors have >|ucount elements.
This word is implemented as C function, which allows us to compare the >performance of the various Forth systems to C.
I have implemented variants of v* in Forth, a straightforward one and
one with pairwise addition. The code can be found in
https://www.complang.tuwien.ac.at/forth/programs/pairwise-fdot.4th
Note that if your Forth system only supports an 8-deep FP stack, you
must not call PAIRWISE-V* unless the FP stack is empty.
I have measured the performance for 10000 dot products of vectors of
50000 elements (so the straightforward algorithm has a minimum
run-time of 0.5G times the latency of F+.
Here are the results (measured on Zen4 (Ryzen 8700G)):
gforth-fast gforth-fast-v3 lxf sf64 vfx64
cycles:u
1_543_071_551 2_044_522_195 v* 6_095_056_325 6_544_418_067 3_221_307_358 4_579_918_381 3_425_243_416 st 4_350_367_901 4_371_526_269 10_527_316_831 10_630_319_712 1_203_870_244 pw
instructions:u
3_618_080_286 3_117_767_051 v*
11_618_280_866 11_617_967_567 9_516_121_645 17_010_999_668 6_074_024_083 st >10_164_410_950 10_164_097_601 14_265_222_850 16_962_950_438 3_823_084_163 pw
The Forth versions are:
gforth-fast current development version
gforth-fast-v3 current development version compiled with -march=x86-64-v3
this makes use of fma instructions
lxf 1.6-982-823
sf64 SwiftForth 4.0.0-RC89
vfx64 5.43
The versions are:
tag word Explanation
v* v* Gforth primitive implemented in C
st naive-v* Straightforward implementation
pw pairwise-v*
VFX64 shows that pairwise addition can be faster than even the C >implementation of the straightforward algorithm (v* in gforth-fast).
VFX also executes slightly fewer instructions than V* on Gforth-fast:
V* has loop overhead, whereas PAIRWISE-V* uses base steps in the
recursion for vectors with up to 64 elements, resulting in less
overhead.
With FMA instructions V* executes fewer instructions (6 per iteration
instead of 7), but more cycles (4 per iteration instead of 3). My >explanation is that the FMA instruction has 4 cycles of latency on
Zen4, while fadd has 3 cycles of latency. Interestingly, in
gforth-fast-v3 NAIVE-V* takes the same number of instructions and one
cycle per iteration more than in gforth-fast, but I have not looked at
why that is.
In gforth-fast the PAIRWISE-V* is quite a bit faster than NAIVE-V*.
It also takes a little fewer instructions, probably because of the
unfolding. But V* is still a lot faster, so our code generation
should be improved.
anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
With FMA instructions V* executes fewer instructions (6 per iteration
instead of 7), but more cycles (4 per iteration instead of 3). My
explanation is that the FMA instruction has 4 cycles of latency on
Zen4, while fadd has 3 cycles of latency.
1. I wonder if FMA is faster on Intel chips.
2. A version using AVX intrinsics would be interesting to compare.
I discussed summing up an array with pairwise addition earlier <2025Jul16.132504@mips.complang.tuwien.ac.at>. The benefits compared
to the straightforward algorithm are:
* Better performance, because the straightforward algorithm is limited
by the dependence chains of the F+ computations.
* Lower rounding errors in the typical case.
The dot-product of two vectors is a primitive in linear algebra, and
also has such a chain of F+ if implemented natively. Implementing the
dot product with pairwise addition should have similar benefits.
Gforth already has a word for the dot product:
|'v*' ( f-addr1 nstride1 f-addr2 nstride2 ucount - r ) gforth-0.5 "v-star"
| dot-product: r=v1*v2. The first element of v1 is at f_addr1, the
|next at f_addr1+nstride1 and so on (similar for v2). Both vectors have |ucount elements.
This word is implemented as C function, which allows us to compare the performance of the various Forth systems to C.
I have implemented variants of v* in Forth, a straightforward one and
one with pairwise addition. The code can be found in
https://www.complang.tuwien.ac.at/forth/programs/pairwise-fdot.4th
Here are the results (measured on Zen4 (Ryzen 8700G)):
gforth-fast gforth-fast-v3 lxf sf64 vfx64
cycles:u
1_543_071_551 2_044_522_195 v*
6_095_056_325 6_544_418_067 3_221_307_358 4_579_918_381 3_425_243_416 st
4_350_367_901 4_371_526_269 10_527_316_831 10_630_319_712 1_203_870_244 pw
instructions:u
3_618_080_286 3_117_767_051 v* 11_618_280_866 11_617_967_567 9_516_121_645 17_010_999_668 6_074_024_083 st 10_164_410_950 10_164_097_601 14_265_222_850 16_962_950_438 3_823_084_163 pw
The Forth versions are:
gforth-fast current development version
gforth-fast-v3 current development version compiled with -march=x86-64-v3
this makes use of fma instructions
lxf 1.6-982-823
sf64 SwiftForth 4.0.0-RC89
vfx64 5.43
- anton
The E-Core beats the P-Core in cycles in relatively many cases; and
even if you take the higher clock rate of the P-Core into account the
E-Core still beats it in time in a number of cases.
Not sure what you mean with "AVX intrinsics" in the context of Forth;
For iForth on AMD Ryzen 7 5800X 8-Core Processor
\ V* : 2,523,563,242 clock ticks
\ naive-V* : 2,532,810,048 clock ticks
\ pairwise-V* : 3,251,776,774 clock ticks
Using recursion to effect compilation inside an interpreted definition
( gen-v* ) is a no-no for iForth because it perceives it as nested >compilation (which is not supported yet).
It is interesting that it
works in so many other Forths.
anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
Not sure what you mean with "AVX intrinsics" in the context of Forth;
Well, your v* primitive is written in C, so it could potentially use AVX >vector operations. _mm512_fmadd_pd to do pointwise multiply-adds into a
YMM register and _mm512_reduce_add_pd to sum the pointwise results at
the end, if I have it right.
perceived as nested definitions.
Are you sure it is that problem? The same technique is used in
GEN-SUM in
https://www.complang.tuwien.ac.at/forth/programs/pairwise-sum.4th
and that works on iForth (at least on the 5.1-mini version I have).
For pairwise-fdot.4th iForth gave me some error that I did not
investigate further; looking at it again, both
iforth 'create testthis include /home/anton/gforth/compat/xlocals.fs include /home/anton/forth/pairwise-fdot.4th test bye'
and
iforth "create benchthis include /home/anton/gforth/compat/xlocals.fs include /home/anton/forth/pairwise-fdot.4th ' pairwise-v* bench bye"
work, so maybe the change that I made in the meantime (some words
generated by GEN-V* used to be normal colon definitions and are now
macros) makes the difference.
Here are the results with iForth, which works with the current
version, on Zen4 (Ryzen 8700G):
gf-f gf-f-v3 iForth lxf sf64 vfx64
k cycles:u
1_543_608 2_043_449 v*
6_090_505 6_544_524 3_399_183 3_219_799 4_705_232 3_425_833 naive-v*
4_355_047 4_360_980 3_850_466 4_368_398 11_998_171 1_208_370 pairwise-v*
Looking at iForth's code for VG*2, I see quite a bit of pushing andI don't see any "VG*2" in https://www.complang.tuwien.ac.at/forth/programs/pairwise-fdot.4th ?
popping going on, so apparently something prevents the register
allocator from working as well as it does on some other code.
On 10/9/2026 7:32 AM, Anton Ertl wrote:
[..]> It does not nest definitions, so it's unclear why it should be
perceived as nested definitions.
The latest iForth checks [ and ] for (not) increasing the code space
in a way that night impact doing nested compilation. It gives a stack
error that looks like a problem with CASE's branch generator . I will >investigate further when I have time. Thanks for reporting.
gf-f gf-f-v3 iForth lxf sf64 vfx64
k cycles:u
1_543_608 2_043_449 v*
6_090_505 6_544_524 3_399_183 3_219_799 4_705_232 3_425_833 naive-v*
4_355_047 4_360_980 3_850_466 4_368_398 11_998_171 1_208_370 pairwise-v*
The most interesting result is the very good result of Gforth's native
v*
and vfx64's surprising sprint in pairwise-v*.
Looking at iForth's code for VG*2, I see quite a bit of pushing andI don't see any "VG*2" in >https://www.complang.tuwien.ac.at/forth/programs/pairwise-fdot.4th ?
popping going on, so apparently something prevents the register
allocator from working as well as it does on some other code.
Strange. I changed fourth and third to what I suppose is worse code, but
\ V* : 2,598,357,958 clock ticks
\ naive-V* : 2,596,436,412 clock ticks
\ pairwise-V* : 2,286,386,056 clock ticks
Now pairwise-V* wins.
fourth and third became:
: fourth ( a b c d -- a b c d a ) >S >S OVER S> S> ROT ;
: third ( a b c -- a b c a ) >S OVER S> SWAP ;
... which looks bad, but which generates
FORTH> ' v*2 idis
$01344740 : v*2
$0134474A pop rbx
$0134474B pop rdi
$0134474C fld [rbx] qword
$0134474E fld [rdi] qword
$01344750 fmulp ST(1), ST
$01344752 pop rax
$01344753 pop rdx
$01344754 fld [rbx rax*1] qword
$01344757 fld [rdx rdi*1] qword
$0134475A fmulp ST(1), ST
$0134475C faddp ST(1), ST
$0134475E fpush,
$01344768 push rdx
$01344769 push rax
$0134476A lea rdi, [rdi rdx*2] qword
$0134476E push rdi
$0134476F lea rbx, [rbx rax*2] qword
$01344773 push rbx
$01344774 ;
peter <peter.noreply@tin.it> writes:
I tested this on lxf64 running on a ryzen 9950x
I got
cycles instructions
pairwise 789,294,432 3,628,952,994
naive 2,510,536,091 6,512,423,204
Nice!
First run with naive I got 35,476,814,233 cycles and 54,012,383,854 instr. >I saw that you checked for F*+ . I have that as a library call, obviously >clang does not use a fma instruction for it but instead compiles a call
to a libray function.
I expect that with the right flags (maybe -march=x86-64-v3 will work)
it will use an fma instruction. Because these instructions were only
added in ~2010, they are not generated normally; and because C's fma() guarantees that rounding only happens at the end (not between the f*
and the f+), it has to do an expensive library function if there is no
fma instruction available.
I would guess that lxf is limited by the use of the x87 fpus small stack >But I need to confirm that
For the version I used in benchmarking, I saw lots of code for the
data stack stuff, so either the FP stuff or working with ]] ... [[
seems to throw the usually perfect code generation of lxf for
straight-line code off track.
- anton
You mean AVX "p"arrallel instructions rather than "s"calar
But given that the stride can be anything, vectorizing V* is not
profitable, so gcc does not do it. One could special-case unit stride
to get at least that case vectorized, but the benchmark I used does
not use unit stride.
You can call the BLAS routine DDOT of libopenblas or some other well-optimized BLAS library and probably get close to optimal code.
marcel hendrix <mhx@iae.nl> writes:
Strange. I changed fourth and third to what I suppose is worse code, but
\ V* : 2,598,357,958 clock ticks
\ naive-V* : 2,596,436,412 clock ticks
\ pairwise-V* : 2,286,386,056 clock ticks
Now pairwise-V* wins.
fourth and third became:
: fourth ( a b c d -- a b c d a ) >S >S OVER S> S> ROT ;
: third ( a b c -- a b c a ) >S OVER S> SWAP ;
... which looks bad, but which generates
FORTH> ' v*2 idis
$01344740 : v*2
$0134474A pop rbx
$0134474B pop rdi
$0134474C fld [rbx] qword
$0134474E fld [rdi] qword
$01344750 fmulp ST(1), ST
$01344752 pop rax
$01344753 pop rdx
$01344754 fld [rbx rax*1] qword
$01344757 fld [rdx rdi*1] qword
$0134475A fmulp ST(1), ST
$0134475C faddp ST(1), ST
$0134475E fpush,
$01344768 push rdx
$01344769 push rax
$0134476A lea rdi, [rdi rdx*2] qword
$0134476E push rdi
$0134476F lea rbx, [rbx rax*2] qword
$01344773 push rbx
$01344774 ;
\ V* : 2,598,357,958 clock ticks
\ naive-V* : 2,596,436,412 clock ticks
\ pairwise-V* : 2,286,386,056 clock ticks
FORTH> BENCH2
\ V* : 2,626,824,105 clock ticks
\ naive-V* : 2,616,912,602 clock ticks
\ pairwise-V* : 2,275,658,158 clock ticks ok
( As V* and naive-V* should give the same result, there is about 0.5% of >clock jitter going on. I did let the caches warm up for a few seconds now. )
Compared to the previous results, with variable stride,
\ V* : 2,598,357,958 clock ticks
\ naive-V* : 2,596,436,412 clock ticks
\ pairwise-V* : 2,286,386,056 clock ticks
It appears the V* code is about 1% slower (!) but pair-wise-V* has
become 1% faster. Decreasing the stack pressure does not help, the pain
is somewhere else.
marcel hendrix <mhx@iae.nl> writes:[..]> VFX generates good code for v*1...v*64, e.g., v*2:
On 10/9/2026 7:32 AM, Anton Ertl wrote:
( 0050AB40 DD03 ) FLD DOUBLE 0 [RBX][..]> VFX compiles 18 Forth words into 15 instructions in this case. For
( 0050AB42 488B5500 ) MOV RDX, [RBP]
( 0050AB46 DD02 ) FLD DOUBLE 0 [RDX]
( 0050AB48 DEC9 ) FMULP ST(1), ST
( 0050AB4A 488B5500 ) MOV RDX, [RBP]
( 0050AB4E 48035510 ) ADD RDX, [RBP+10]
( 0050AB52 48035D08 ) ADD RBX, [RBP+08]
( 0050AB56 DD03 ) FLD DOUBLE 0 [RBX]
( 0050AB58 DD02 ) FLD DOUBLE 0 [RDX]
( 0050AB5A DEC9 ) FMULP ST(1), ST
( 0050AB5C 48035510 ) ADD RDX, [RBP+10]
( 0050AB60 48035D08 ) ADD RBX, [RBP+08]
( 0050AB64 DEC1 ) FADDP ST(1), ST
( 0050AB66 48895500 ) MOV [RBP], RDX
( 0050AB6A C3 ) RET/NEXT
( 43 bytes, 15 instructions )
v*64, it compiles 576 Forth words into 387 instructions. It may or
may not be helpful for performance to load the values that it accesses
at [RBP+10] and [RBP+08] into registers, too, to reduce the usage of
load ports, which may limit the performance of this code.
- anton
On 10/9/2026 9:20 AM, Anton Ertl wrote:
marcel hendrix <mhx@iae.nl> writes:[..]> VFX generates good code for v*1...v*64, e.g., v*2:
On 10/9/2026 7:32 AM, Anton Ertl wrote:
[..]> VFX compiles 18 Forth words into 15 instructions in this case. For
( 0050AB40 DD03 ) FLD DOUBLE 0 [RBX]
( 0050AB42 488B5500 ) MOV RDX, [RBP]
( 0050AB46 DD02 ) FLD DOUBLE 0 [RDX]
( 0050AB48 DEC9 ) FMULP ST(1), ST
( 0050AB4A 488B5500 ) MOV RDX, [RBP]
( 0050AB4E 48035510 ) ADD RDX, [RBP+10]
( 0050AB52 48035D08 ) ADD RBX, [RBP+08]
( 0050AB56 DD03 ) FLD DOUBLE 0 [RBX]
( 0050AB58 DD02 ) FLD DOUBLE 0 [RDX]
( 0050AB5A DEC9 ) FMULP ST(1), ST
( 0050AB5C 48035510 ) ADD RDX, [RBP+10]
( 0050AB60 48035D08 ) ADD RBX, [RBP+08]
( 0050AB64 DEC1 ) FADDP ST(1), ST
( 0050AB66 48895500 ) MOV [RBP], RDX
( 0050AB6A C3 ) RET/NEXT
( 43 bytes, 15 instructions )
v*64, it compiles 576 Forth words into 387 instructions. It may or
may not be helpful for performance to load the values that it accesses
at [RBP+10] and [RBP+08] into registers, too, to reduce the usage of
load ports, which may limit the performance of this code.
VFX is about 3 times faster for paired-v*, which reeks of some magical >loophole in the x86 instruction set ... It benefits from (1) not >push/popping RBX and RBX (assuming the FADDP doesn't swamp that) and (2)
not saving FTOS to memory after the FADDP . I wonder if (2) is by
design, or if it simply assumes (or makes sure) the FP-stack will be
deep enough for further iterations. (I found "making sure" too difficult >without interrupts.)
With VFX64 5.43, was FPSYSTEM value == 1?
anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
But given that the stride can be anything, vectorizing V* is not
profitable, so gcc does not do it. One could special-case unit stride
to get at least that case vectorized, but the benchmark I used does
not use unit stride.
Maybe it would also benefit at stride=2 or (especially in single
precision) stride=4.
I didn't realize stride!=1 was commonly used for
dot-product-like operations.
Paul Rubin <no.email@nospam.invalid> writes: >anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
Not sure what you mean with "AVX intrinsics" in the context of Forth;
Well, your v* primitive is written in C, so it could potentially use AVX >vector operations. _mm512_fmadd_pd to do pointwise multiply-adds into a >YMM register and _mm512_reduce_add_pd to sum the pointwise results at
the end, if I have it right.
You mean AVX "p"arrallel instructions rather than "s"calar
instructions used by the generated code, in order to vectorize the code.
Actually, if V* used unit stride and was compiled with -ffast-math
(which allows the compiler to pretend that the associative law holds
for FP addition and multiplication, among other things), gcc would
vectorize V* and, with -march=x86-64-v3 use the AVX instructions corresponding to these intrinsics, so one gets the result without
using intrinsics.
But given that the stride can be anything, vectorizing V* is not
profitable, so gcc does not do it. One could special-case unit stride
to get at least that case vectorized, but the benchmark I used does
not use unit stride.
Even in the vectorized case, the dependency between the FP additions
would be a problem; the compiler could unroll the loop and (thanks to -ffast-math) reassociate the additions from
... f* f+ ... f* f+ ... f* f+
where every F+ depends on the one before to
... f* ... f* f+ ... f* f+ f+
i.e., only the last f+ depends on the previous iteration, while the
other two are in a new dependence chain. And if unrolling by a factor
of 3 is not enough, use more unrolling until the loop execution time
is dominated by resource consumption rather than latency, but my
impression of the gcc- and clang-generated code is that the versions I
looked at are not doing that.
You can call the BLAS routine DDOT of libopenblas or some other well-optimized BLAS library and probably get close to optimal code.
- anton
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 76 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 55:03:14 |
| Calls: | 1,265 |
| Calls today: | 1 |
| Files: | 1,362 |
| D/L today: |
3 files (2,684K bytes) |
| Messages: | 285,585 |