• Re: FYI, build times for 15.0-RELEASE and 14.4-RELEASE poudriere jail worlds vs. 14.3-RELEASE poudriere jail worlds: an example [14.4/14.3 time ratio varies notably for WInDevKit2023 vs. 7950X]

    From Mark Millard@marklmi@yahoo.com to muc.lists.freebsd.stable on Sun Apr 19 07:41:43 2026
    From Newsgroup: muc.lists.freebsd.stable

    On 4/18/26 17:02, Mark Millard wrote:
    On 4/18/26 12:26, Mark Millard wrote:
    There have been reports of rather large time differences for
    14.4-RELEASE vs. 14.3-RELEASE --as well as prior reports referencing
    14.3-RELEASE vs. 15.0-RELEASE. This is just a report of from-scratch
    building of a particular set of port-packages on a fairly fast machine
    (amd64 7950X3D) for each of those.

    Upstream based context installed/used:

    pkgbase GENERIC-NODEBUG main boot kernel
    pkgbase debug main boot world (/etc/malloc.conf -> junk:false)

    pkgbase 15.0-RELEASE poudriere-devel jail world.
    ftp-archive 14.4-RELEASE poudriere-devel jail world.
    ftp-archive 14.3-RELEASE poudriere-devel jail world.

    (So: None are my builds.)

    # poudriere jail -l
    JAILNAME VERSION OSVERSION ARCH METHOD
    TIMESTAMP PATH
    release14p3-amd64 14.3-RELEASE-p10 1403000 amd64 ftp-archive
    2026-04-14 18:00:43 /usr/local/poudriere/jails/release14p3-amd64
    . . .
    release14-amd64 14.4-RELEASE-p1 1404000 amd64 ftp-archive
    2026-04-14 18:02:42 /usr/local/poudriere/jails/release14-amd64
    . . .
    release-amd64 15.0-RELEASE-p5 1500068 amd64 pkgbase
    2026-04-17 16:09:49 /usr/local/poudriere/jails/release-amd64
    . . .

    ZFS context.


    As for the 7950X3D build times:
    (maximum ratio: 02:29:18/02:04:06 approx.= 1.2)

    15.0-RELEASE (run third) took the most time:
    (1 less built than for 14.*-RELEASE : 687 vs. 688)

    [02:29:20] [release-amd64-alt] [2026-04-18_08h47m48s] [committing] Time:
    02:29:18
    Queued: 688 Inspected: 0 Ignored: 0 Built: 687 Failed: 1
    Skipped: 0 Fetched: 0 Remaining: 0


    14.4-RELEASE (run second) took the middle amount of time:

    [02:24:13] [release14-amd64-alt] [2026-04-18_06h23m35s] [committing]
    Time: 02:24:11
    Queued: 689 Inspected: 0 Ignored: 0 Built: 688 Failed: 1
    Skipped: 0 Fetched: 0 Remaining: 0

    Note: 02:24:11/02:04:05 approx.= 1.16
    Note: max ratio difference: 1.2 - 1.16 = 0.04



    14.3-RELEASE (run first) took the least time, by a bigger amount:

    [02:04:06] [release14p3-amd64-alt] [2026-04-17_21h53m02s] [committing]
    Time: 02:04:05
    Queued: 689 Inspected: 0 Ignored: 0 Built: 688 Failed: 1
    Skipped: 0 Fetched: 0 Remaining: 0


    So, in the 79050X3D's context, 14.4-RELEASE and 15.0-RELEASE are similar
    in how much more time they take than 14.3-RELEASE.

    (Wording above intended as via time ratio comparisons to 14.3.)

    Note: The above 14.4 vs. 15.0 comparison/contrast was not true for the WinDevKit20203 (aarch64) of the 3 releases via the ratios, see later
    below. So 15.0 has something rather significant that makes builds take
    more time on the WinDevKit20203 compared to 14.4's ratio.

    (The WinDevKit20203 was tested with a smaller subset of the
    port-packages to be built but still took more than a day to finish the
    sequence of testing 3 poudriere-devel jail releases.)

    This suggests that
    the biggest difference makers are probably just differences between 14.4
    and 14.3 that are also in 15.0, not any extra differences 15.0 may have
    vs. 14.3.

    Note: Again, not true for the WinDevKit2023 15.0-RELEASE vs.
    14.4-RELEASE differences.


    I've never managed to replicate the much larger 14.4 vs. 14.3 time
    ratios that have been reported.

    Both platforms were somewhat under 1.25 for the maximum ratio. That is
    much less than one source of build time performance reports that I've seen.



    For reference:

    # ~/fbsd-based-on-what-commit.sh -C /usr/ports-alt/
    b96a271bbc08 (HEAD -> main, freebsd/main, freebsd/HEAD) www/firefox:
    update to 150.0 (rc1)
    Author: Christoph Moench-Tegeder <cmt@FreeBSD.org>
    Commit: Christoph Moench-Tegeder <cmt@FreeBSD.org>
    CommitDate: 2026-04-16 17:51:06 +0000
    branch: main
    merge-base: b96a271bbc0820a058e89001c9ac2309cbf8a60d
    merge-base: CommitDate: 2026-04-16 17:51:06 +0000
    n742012 (--first-parent --count for merge-base)

    /usr/ports-alt/ is a clean tree.

    My /usr/local/etc/poudriere.d/poudriere.conf is involved and likely unusual. >>
    15.0-RELEASE has a newer jemalloc.


    The below paragraph spans the 3 release tests:

    The SWAP space was never put to use. The 1min load average got to be as
    high as: 178.27, the 5min: 125.95, the 15min: 76.14 . (Sampled every top
    update by a patched top.) There are 16 SMT cores, so 32 FreeBSD cpus.
    Implicit 32 builders allowed in parallel. ALLOW_MAKE_JOBS=yes was in
    use but no MAKE_JOBS_NUMBER_LIMIT or the like was in use, so: implicit
    32 per builder allowed. The media in use was (is): PCIe Optane 1.4 TB.

    Note: 192 GiBytes of RAM.




    SIDE NOTE:
    A different example, aarch64 Windows Dev Kit 2023 context, USB3 media,
    UFS, only around 273 port-packages queued/built . . .

    maximum ratio: 09:58:30/08:10:58 approx.= 1.22
    (That was for 15.0-RELEASE)

    But that is a lot more than for 14.4 ratio for WinDevKit2023, unlike
    what was observed for the 7950X3D.



    15.0-RELEASE:

    This was run third and the in-process from-scratch build queued 271 (2
    less than under 14.*). It will be many hours before there is a final
    "time" for this.


    UPDATE 15.0-RELEASE NOTES LATER, given how long the WinDevKit2023 builds
    take.

    [09:59:09] [release-aarch64-alt] [2026-04-18_15h34m27s] [committing]
    Time: 09:58:30
    Queued: 271 Inspected: 0 Ignored: 0 Built: 271 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    This is a lot longer than 14.4 below. So it significantly contributes to
    the maximum ratio vs. 14.3 for the WinDevKit2023 test.



    14.4-RELEASE (run first):

    [09:12:00] [release14-aarch64-alt] [2026-04-17_22h10m56s] [committing]
    Time: 09:11:36
    Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    Note: 09:11:36/08:10:58 approx.= 1.12
    Note: max ratio difference: 1.22 - 1.12 = 0.10 (unlike 7950X3D 0.04)



    14.3-RELEASE:

    This was run second.


    UPDATE 14.3-RELEASE NOTES LATER, given how long the WinDevKit2023 builds
    take.

    [08:11:31] [release14p3-aarch64-alt] [2026-04-18_07h22m56s] [committing] Time: 08:10:58
    Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    Note: 09:11:36/08:10:58 approx.= 1.12

    Other notes spanning the 3 release tests:

    4208Mi was the maximum swap space use observed, there being 32 GiBytes
    of RAM. The 1min load average got to be as high as: 61.29, the 5min:
    55.65, the 15min: 48.65 . (Sampled every top update by a patched top.)
    There are 8 single-hardware-thread cores, so 8 FreeBSD cpus. Implicit 8 builders allowed in parallel. ALLOW_MAKE_JOBS=yes was in use but no MAKE_JOBS_NUMBER_LIMIT or the like was in use, so: implicit 8 per
    builder allowed. The media in use was (is): U.2 Optane 1.4 TB via a USB3 adapter.
    --
    ===
    Mark Millard
    marklmi at yahoo.com


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21f-Linux NewsLink 1.2
  • From Mark Millard@marklmi@yahoo.com to muc.lists.freebsd.stable on Sun Apr 19 14:50:19 2026
    From Newsgroup: muc.lists.freebsd.stable

    On 4/19/26 07:41, Mark Millard wrote:
    On 4/18/26 17:02, Mark Millard wrote:
    On 4/18/26 12:26, Mark Millard wrote:
    There have been reports of rather large time differences for
    14.4-RELEASE vs. 14.3-RELEASE --as well as prior reports referencing
    14.3-RELEASE vs. 15.0-RELEASE. This is just a report of from-scratch
    building of a particular set of port-packages on a fairly fast machine
    (amd64 7950X3D) for each of those.

    Upstream based context installed/used:

    pkgbase GENERIC-NODEBUG main boot kernel
    pkgbase debug main boot world (/etc/malloc.conf -> junk:false) >>>
    pkgbase 15.0-RELEASE poudriere-devel jail world.
    ftp-archive 14.4-RELEASE poudriere-devel jail world.
    ftp-archive 14.3-RELEASE poudriere-devel jail world.

    (So: None are my builds.)

    # poudriere jail -l
    JAILNAME VERSION OSVERSION ARCH METHOD
    TIMESTAMP PATH
    release14p3-amd64 14.3-RELEASE-p10 1403000 amd64 ftp-archive
    2026-04-14 18:00:43 /usr/local/poudriere/jails/release14p3-amd64
    . . .
    release14-amd64 14.4-RELEASE-p1 1404000 amd64 ftp-archive
    2026-04-14 18:02:42 /usr/local/poudriere/jails/release14-amd64
    . . .
    release-amd64 15.0-RELEASE-p5 1500068 amd64 pkgbase
    2026-04-17 16:09:49 /usr/local/poudriere/jails/release-amd64
    . . .

    ZFS context.

    ANother note about my context: the defaults for gcc and llvm are:

    lang/gcc15
    devel/llvm22

    but the 7905X3D also built:

    lang/gcc14
    devel/llvm19

    The port-packages that WinDevKit2023 built did not include lang/gcc14 or devel/llvm19 .



    As for the 7950X3D build times:
    (maximum ratio: 02:29:18/02:04:06 approx.= 1.2)

    15.0-RELEASE (run third) took the most time:
    (1 less built than for 14.*-RELEASE : 687 vs. 688)

    [02:29:20] [release-amd64-alt] [2026-04-18_08h47m48s] [committing] Time: >>> 02:29:18
    Queued: 688 Inspected: 0 Ignored: 0 Built: 687 Failed: 1
    Skipped: 0 Fetched: 0 Remaining: 0


    14.4-RELEASE (run second) took the middle amount of time:

    [02:24:13] [release14-amd64-alt] [2026-04-18_06h23m35s] [committing]
    Time: 02:24:11
    Queued: 689 Inspected: 0 Ignored: 0 Built: 688 Failed: 1
    Skipped: 0 Fetched: 0 Remaining: 0

    Note: 02:24:11/02:04:05 approx.= 1.16
    Note: max ratio difference: 1.2 - 1.16 = 0.04



    14.3-RELEASE (run first) took the least time, by a bigger amount:

    [02:04:06] [release14p3-amd64-alt] [2026-04-17_21h53m02s] [committing]
    Time: 02:04:05
    Queued: 689 Inspected: 0 Ignored: 0 Built: 688 Failed: 1
    Skipped: 0 Fetched: 0 Remaining: 0


    So, in the 79050X3D's context, 14.4-RELEASE and 15.0-RELEASE are similar >>> in how much more time they take than 14.3-RELEASE.

    (Wording above intended as via time ratio comparisons to 14.3.)

    Note: The above 14.4 vs. 15.0 comparison/contrast was not true for the WinDevKit20203 (aarch64) of the 3 releases via the ratios, see later
    below. So 15.0 has something rather significant that makes builds take
    more time on the WinDevKit20203 compared to 14.4's ratio.

    (The WinDevKit20203 was tested with a smaller subset of the
    port-packages to be built but still took more than a day to finish the sequence of testing 3 poudriere-devel jail releases.)

    This suggests that
    the biggest difference makers are probably just differences between 14.4 >>> and 14.3 that are also in 15.0, not any extra differences 15.0 may have
    vs. 14.3.

    Note: Again, not true for the WinDevKit2023 15.0-RELEASE vs.
    14.4-RELEASE differences.


    I've never managed to replicate the much larger 14.4 vs. 14.3 time
    ratios that have been reported.

    Both platforms were somewhat under 1.25 for the maximum ratio. That is
    much less than one source of build time performance reports that I've seen.



    For reference:

    # ~/fbsd-based-on-what-commit.sh -C /usr/ports-alt/
    b96a271bbc08 (HEAD -> main, freebsd/main, freebsd/HEAD) www/firefox:
    update to 150.0 (rc1)
    Author: Christoph Moench-Tegeder <cmt@FreeBSD.org>
    Commit: Christoph Moench-Tegeder <cmt@FreeBSD.org>
    CommitDate: 2026-04-16 17:51:06 +0000
    branch: main
    merge-base: b96a271bbc0820a058e89001c9ac2309cbf8a60d
    merge-base: CommitDate: 2026-04-16 17:51:06 +0000
    n742012 (--first-parent --count for merge-base)

    /usr/ports-alt/ is a clean tree.

    My /usr/local/etc/poudriere.d/poudriere.conf is involved and likely unusual.

    15.0-RELEASE has a newer jemalloc.


    The below paragraph spans the 3 release tests:

    The SWAP space was never put to use. The 1min load average got to be as
    high as: 178.27, the 5min: 125.95, the 15min: 76.14 . (Sampled every top >>> update by a patched top.) There are 16 SMT cores, so 32 FreeBSD cpus.
    Implicit 32 builders allowed in parallel. ALLOW_MAKE_JOBS=yes was in
    use but no MAKE_JOBS_NUMBER_LIMIT or the like was in use, so: implicit
    32 per builder allowed. The media in use was (is): PCIe Optane 1.4 TB.

    Note: 192 GiBytes of RAM.




    SIDE NOTE:
    A different example, aarch64 Windows Dev Kit 2023 context, USB3 media,
    UFS, only around 273 port-packages queued/built . . .

    maximum ratio: 09:58:30/08:10:58 approx.= 1.22
    (That was for 15.0-RELEASE)

    Later I show use of more significant digits that I did originally.


    But that is a lot more than for 14.4 ratio for WinDevKit2023, unlike
    what was observed for the 7950X3D.



    15.0-RELEASE:

    This was run third and the in-process from-scratch build queued 271 (2
    less than under 14.*). It will be many hours before there is a final
    "time" for this.


    UPDATE 15.0-RELEASE NOTES LATER, given how long the WinDevKit2023 builds >>> take.

    [09:59:09] [release-aarch64-alt] [2026-04-18_15h34m27s] [committing]
    Time: 09:58:30
    Queued: 271 Inspected: 0 Ignored: 0 Built: 271 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    This is a lot longer than 14.4 below. So it significantly contributes to
    the maximum ratio vs. 14.3 for the WinDevKit2023 test.



    14.4-RELEASE (run first):

    [09:12:00] [release14-aarch64-alt] [2026-04-17_22h10m56s] [committing]
    Time: 09:11:36
    Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    Note: 09:11:36/08:10:58 approx.= 1.12
    Note: max ratio difference: 1.22 - 1.12 = 0.10 (unlike 7950X3D 0.04)



    14.3-RELEASE:

    This was run second.


    UPDATE 14.3-RELEASE NOTES LATER, given how long the WinDevKit2023 builds >>> take.

    [08:11:31] [release14p3-aarch64-alt] [2026-04-18_07h22m56s] [committing]
    Time: 08:10:58
    Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    Note: 09:11:36/08:10:58 approx.= 1.12

    Other notes spanning the 3 release tests:

    4208Mi was the maximum swap space use observed, there being 32 GiBytes
    of RAM. The 1min load average got to be as high as: 61.29, the 5min:
    55.65, the 15min: 48.65 . (Sampled every top update by a patched top.)
    There are 8 single-hardware-thread cores, so 8 FreeBSD cpus. Implicit 8 builders allowed in parallel. ALLOW_MAKE_JOBS=yes was in use but no MAKE_JOBS_NUMBER_LIMIT or the like was in use, so: implicit 8 per
    builder allowed. The media in use was (is): U.2 Optane 1.4 TB via a USB3 adapter.



    In order to make the time ratio comparisons/contrasts more reasonable,
    the below is based on the 7950X3D building the same port-packages as the WinDevKit2023 built:

    15.0-RELEASE (built first) build time:

    [01:18:43] [release-amd64-alt] [2026-04-19_09h54m55s] [committing] Time: 01:18:41
    Queued: 271 Inspected: 0 Ignored: 0 Built: 271 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    15.0 vs. 14.3 time ratio: 01:18:41/01:06:52 approx.= 1.177


    14.4-RELEASE (built second) build time:

    [01:16:20] [release14-amd64-alt] [2026-04-19_11h13m39s] [committing]
    Time: 01:16:18
    Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0

    14.4 vs. 14.3 time ratio: 01:16:18/01:06:52 approx.= 1.147


    14.3-RELEASE (built third) build time:

    [01:06:53] [release14p3-amd64-alt] [2026-04-19_12h29m59s] [committing]
    Time: 01:06:52
    Queued: 273 Inspected: 0 Ignored: 0 Built: 273 Failed: 0
    Skipped: 0 Fetched: 0 Remaining: 0


    [Note: overall max observed load averages spanning the 3 7950X3D builds:
    1min: 96.84, 5min: 61.24, 15min: 51.75 ]


    So, for the comparison/contrast based on the same port-packages being built:

    7950X3D 15.0 and 14.4 vs. 14.3 ratios:
    15.0: 01:18:41/01:06:52 approx.= 1.177
    14.4: 01:16:18/01:06:52 approx.= 1.147
    So: 1.177 - 1.147 approx.= 0.030
    (so 14.4 is a far bigger ratio increment in the sequence: 0.147)
    (sequence: 14.3 -> 14.4 -> 15.0)

    WinDevKit2023 15.0 and 14.4 vs. 14.3 ratios:
    15.0: 09:58:30/08:10:58 approx.= 1.219
    14.4: 09:11:36/08:10:58 approx.= 1.121
    1.219 - 1.121 approx.= 0.088
    (so 14.4 is a bigger ratio increment in the sequence as well: 0.121)
    (sequence: 14.3 -> 14.4 -> 15.0)

    In both cases, it looks like focusing on what is different in 14.4 vs.
    14.3 that is also different in 15.0 vs. 14.3 would be focusing on the
    bigger build-time-taking issues, presuming updating both 15.* and 14.*
    for whatever helps.

    WITH_LLVM_LINK_STATIC_LIBRARIES could well be involved.
    --
    ===
    Mark Millard
    marklmi at yahoo.com


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21f-Linux NewsLink 1.2