From Newsgroup: muc.lists.freebsd.ports

I need to run poudriere and other dev things on a arm64 machine with
only 16GB or RAM.
It can build llvm, but it can't build rust*, chromium, and some other extra-large packages.
Poudriere in QEMU can't build rust, and not even python312.
I need rust-1.98.1.pkg for 15/aarch64 and 16/aarch64.
The rust-1.98.1.pkg for 16 is ready in the builder but the builder is
slow and it would take 5 more days to finish the batch.

Questions:
1. Is it possible to download built packages directly from builder servers?
2. Is there a clean way to add such packages into Poudriere? Do I drop
the package file into the packages/**/All folder?



Thanks,
Yuri


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

On 9/15/26 16:50, Yuri wrote:

I need to run poudriere and other dev things on a arm64 machine with
only 16GB or RAM.
It can build llvm, but it can't build rust*, chromium, and some other extra-large packages.

At least for lang/rust , it seems likely that there is more to the
story, such as how to configure such that rust builds with less RAM.

) Total space: RAM+SWAP ?

) UFS vs. ZFS ?

) What value of vm.pageout_oom_seq ?

) How many FreeBSD CPUs? How many parallel builders?

) Do you use ALLOW_MAKE_JOBS= with no MAKE_JOBS_NUMBER_LIMIT or the
like? (Or: ALLOW_MAKE_JOBS_PACKAGES ?) What value?

) Do you have USE_TMPFS=all (or other RAM+SWAP intensive values)
--but not use TMPFS_BACKLIST to avoid tmpfs requiring huge RAM_SWAP
all to itself for rust? (The file system space is 30 GiBytes these
days as I remember.)

) Do you use MUTUALLY_EXCLUSIVE_BUILD_PACKAGES (or just a single
builder)?

) Any non-default LTO use?

Sorry for the age (early 2024) of the below 2 GiByte RAM armv7 example
but it is the most recent such "small RAM" experiment that I found in my records . . .

More than rust was built. It had 3.6 GiBytes of SWAP (so 5.6 GiBytes==RAM+SWAP), in a UFS context (no ZFS in use). Also, this was
doing my normal armv7 (and aarch64) style of devel/llvm* build: OPTION'd
to BE_NATIVE instead of BE_STANDARD and OPTION'd to not build MLIR. 4
FreeBSD CPUs (i.e, 4 Cortext-A7 cores).

USE_TMPFS=no
PARALLEL_JOBS=2
ALLOW_MAKE_JOBS=yes
MAKE_JOB_NUMBER_LIMIT=2
MAX_EXECUTION_TIME=432000
NOHANG_TIME=432000
MAX_EXECUTION_TIME_EXTRACT=14400
MAX_EXECUTION_TIME_INSTALL=14400
MAX_EXECUTION_TIME_PACKAGE=57600
MAX_EXECUTION_TIME_DEINSTALL=14400

Also the sysctl: vm.pageout_oom_seq=120

The "what builds took over an hour" summary from the 265 packages built was:

<[01:51:31] [01] [01:00:07] Finished lang/perl5.36 | perl5-5.36.3_1: Success [08:55:35] [02] [03:08:09] Finished devel/icu | icu-74.2,1: Success
[13:17:38] [02] [01:28:32] Finished lang/ruby31 | ruby-3.1.4_1,1: Success [14:17:44] [01] [09:20:55] Finished devel/cmake-core |
cmake-core-3.28.3: Success
[4D:01:03:43] [02] [3D:08:48:53] Finished lang/rust | rust-1.76.0: Success [4D:06:26:24] [02] [03:09:35] Finished devel/binutils@native | binutils-2.40_5,1: Success
[4D:14:54:31] [02] [03:38:55] Finished devel/aarch64-none-elf-gcc | aarch64-none-elf-gcc-11.3.0_3: Success
[4D:16:13:00] [01] [4D:01:55:03] Finished devel/llvm18@default | llvm18-18.1.0.r3: Success
[4D:18:05:58] [02] [03:11:00] Finished devel/arm-none-eabi-gcc | arm-none-eabi-gcc-11.3.0_3: Success
[4D:23:00:13] [01] [06:46:06] Finished devel/boost-libs |
boost-libs-1.84.0: Success
[5D:00:16:39] [01] [01:15:53] Finished textproc/source-highlight | source-highlight-3.1.9_9: Success
[5D:01:17:24] [02] [07:10:52] Finished lang/gcc13 | gcc13-13.2.0_4: Success [5D:09:38:14] [01] [05:56:48] Finished devel/freebsd-gcc13@armv7 | armv7-gcc13-13.2.0_1: Success
[5D:10:18:58] [02] [05:44:02] Finished devel/gdb@py39 | gdb-14.1_2: Success [5D:10:31:56] Stopping 2 builders
[main-CA7-default] [2024-03-06_03h15m10s] [committing] Queued: 265
Built: 265 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 Tobuild: 0
Time: 5D:10:31:55>

So, the rust build happened during the llvm18 build:

[4D:01:03:43] [02] [3D:08:48:53] (rust-1.76.0)
[4D:16:13:00] [01] [4D:01:55:03] (llvm18-18.1.0.r3)

and both completed.

For reference: 2794Mi MaxObs(Act+Wir+Lndry+SwapUsed) ("MaxObs": short
for "Maximum Observed"). So, about half of the 5.6 GiBytes RAM+SWAP.

I expect rust can still be built with less than 8 GiBytes of RAM --but
I've no modern evidence. There might be other properties of such a build
that are not nice, but that would be a different issue.

With aarch64, SWAP<=3.6*RAM avoids warnings about potential mis-tuning.
So RAM+SWAP=4.6*RAM would be an example, over 36 GiBytes for RAM+SWAP
for 8 GiBytes of RAM.

(I always avoid "spinning rust" media being involved.)

Poudriere in QEMU can't build rust, and not even python312.
I need rust-1.98.1.pkg for 15/aarch64 and 16/aarch64.
The rust-1.98.1.pkg for 16 is ready in the builder but the builder is
slow and it would take 5 more days to finish the batch.

Questions:
1. Is it possible to download built packages directly from builder servers? 2. Is there a clean way to add such packages into Poudriere? Do I drop
the package file into the packages/**/All folder?

Thanks,
Yuri

--
===
Mark Millard
marklmi at yahoo.com


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

On 9/15/26 22:08, Mark Millard wrote:

I expect rust can still be built with less than 8 GiBytes of RAM --but
I've no modern evidence. There might be other properties of such a build
that are not nice, but that would be a different issue.

Ok,


I will try to hack it to compile in <8 GB.


Yuri



--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

------=_Part_674_610223805.1789546418568
Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit

Hi,

Rust builds fine on my Raspberry Pi 5 with 16 GB RAM. It took about 14 hours. The build does not use qemu so I'm wondering. I've never tried to build chromium.

I think you should take a good look at the error message. Poudriere has some settings that can be tuned, like the use of TMPFS (I disabled that, at least for rust). Also check how many parallel jobs poudriere is running. Combined with the number of CPUs one job is allowed to use. For example compiling llvm together with rust would make it all use a lot of resources and take a much longer time.

Somebody else already answered with the -b option to poudriere. This works as long as your ports tree is older than the latest results of the official builders. If rust is updated in ports, but the official ports builders didn't finish yet, your machine will still build rust again.

I hope this answer helps you a bit.

Regards,
Ronald.


Van: Yuri <yuri@FreeBSD.org>
Datum: woensdag, 16 september 2026 01:50
Aan: "ports@freebsd.org" <ports@freebsd.org>
Onderwerp: Can I download packages from the builder?

I need to run poudriere and other dev things on a arm64 machine with only 16GB or RAM.
It can build llvm, but it can't build rust*, chromium, and some other extra-large packages.
Poudriere in QEMU can't build rust, and not even python312.
I need rust-1.98.1.pkg for 15/aarch64 and 16/aarch64.
The rust-1.98.1.pkg for 16 is ready in the builder but the builder is slow and it would take 5 more days to finish the batch.

Questions:
1. Is it possible to download built packages directly from builder servers? 2. Is there a clean way to add such packages into Poudriere? Do I drop the package file into the packages/**/All folder?

Thanks,
Yuri

------=_Part_674_610223805.1789546418568
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<html><head></head><body>Hi,<br>

Rust builds fine on my Raspberry Pi 5 with 16 GB RAM. It took about 14 hours. The build does not use qemu so I'm wondering. I've never tried to build chromium.<br>

I think you should take a good look at the error message. Poudriere has some settings that can be tuned, like the use of TMPFS (I disabled that, at least for rust). Also check how many parallel jobs poudriere is running. Combined with the number of CPUs one job is allowed to use. For example compiling llvm together with rust would make it all use a lot of resources and take a much longer time.<br>

Somebody else already answered with the -b option to poudriere. This works as long as your ports tree is older than the latest results of the official builders. If rust is updated in ports, but the official ports builders didn't finish yet, your machine will still build rust again.<br>

I hope this answer helps you a bit.<br>

Regards,<br>
Ronald.<br>

&nbsp;
<p><strong>Van:</strong> Yuri &lt;yuri@FreeBSD.org&gt;<br> <strong>Datum:</strong> woensdag, 16 september 2026 01:50<br> <strong>Aan:</strong> "ports@freebsd.org" &lt;ports@freebsd.org&gt;<br> <strong>Onderwerp:</strong> Can I download packages from the builder?</p>

<blockquote style="padding-right: 0px; padding-left: 5px; margin-left: 5px; border-left: #000000 2px solid; margin-right: 0px">
<div class="MessageRFC822Viewer" id="P">
<div class="TextPlainViewer" id="P.P">I need to run poudriere and other dev things on a arm64 machine with only 16GB or RAM.<br>
It can build llvm, but it can't build rust*, chromium, and some other extra-large packages.<br>
Poudriere in QEMU can't build rust, and not even python312.<br>
I need rust-1.98.1.pkg for 15/aarch64 and 16/aarch64.<br>
The rust-1.98.1.pkg for 16 is ready in the builder but the builder is slow and it would take 5 more days to finish the batch.<br>

Questions:<br>
1. Is it possible to download built packages directly from builder servers?<br> 2. Is there a clean way to add such packages into Poudriere? Do I drop the package file into the packages/**/All folder?<br>



Thanks,<br>
Yuri<br>
&nbsp;</div>

<hr></div>
</blockquote>

&nbsp;</body></html>
------=_Part_674_610223805.1789546418568--


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

On 9/16/26 03:13, Ronald Klop wrote:

Somebody else already answered with the -b option to poudriere. This
works as long as your ports tree is older than the latest results of the official builders. If rust is updated in ports, but the official ports builders didn't finish yet, your machine will still build rust again.

Thanks to the -b flag, most of my amd64 builds have been running on an
old 4-core gaming machine with 16 GiB RAM (USE_TMPFS=no, plenty of
swap). I don't often have to fire up my noisy 300 watt PowerEdge anymore.

Since one of the monster ports (llvm, rust, chromium, etc.) is likely to
be updated on any given day, I just avoid updating my poudriere ports
tree too frequently. I update once every few days, which gives the
package build farm good odds of producing binaries for the versions in
my ports tree before I need them. I don't let the tree get much older
than this, though, to ensure validity of my tests.

If I see a monster port building, I might actually kill the build and
try again a day later, especially if I have other ports to test in the meantime. They do build, but tie up the machine all day.
--
Life is a game. Play hard. Play fair. Have fun.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

On 9/15/26 22:59, Yuri wrote:

On 9/15/26 22:08, Mark Millard wrote:

I expect rust can still be built with less than 8 GiBytes of RAM --but
I've no modern evidence. There might be other properties of such a build
that are not nice, but that would be a different issue.

Ok,

I will try to hack it to compile in <8 GB.

I should have noted that my wording presumed use of poudriere-devel
instead of poudriere: it likely referenced newer poudriere notation for
newer functionality and I use only poudriere-devel .

Yuri

--
===
Mark Millard
marklmi at yahoo.com


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

--00000000000044a7dd065bb35bea
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

El mi=C3=A9, 16 sept 2026, 1:51, Yuri <yuri@freebsd.org> escribi=C3=B3:

I need to run poudriere and other dev things on a arm64 machine with
only 16GB or RAM.
It can build llvm, but it can't build rust*, chromium, and some other extra-large packages.
Poudriere in QEMU can't build rust, and not even python312.
I need rust-1.98.1.pkg for 15/aarch64 and 16/aarch64.
The rust-1.98.1.pkg for 16 is ready in the builder but the builder is
slow and it would take 5 more days to finish the batch.

Questions:
1. Is it possible to download built packages directly from builder server=

s?


Have you tried this?

https://github.com/freebsd/poudriere/blob/8b41a68f93647529950036c85f73c7f90= ad72a07/src/etc/poudriere.conf.sample#L386

2. Is there a clean way to add such packages into Poudriere? Do I drop

the package file into the packages/**/All folder?

Thanks,
Yuri

--00000000000044a7dd065bb35bea
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"auto"><div><br><br><div class=3D"gmail_quote gmail_quote_contai= ner"><div dir=3D"ltr" class=3D"gmail_attr">El mi=C3=A9, 16 sept 2026, 1:51,=
Yuri &lt;<a href=3D"mailto:yuri@freebsd.org">yuri@freebsd.org</a>&gt; escr= ibi=C3=B3:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
.8ex;border-left:1px #ccc solid;padding-left:1ex">I need to run poudriere = and other dev things on a arm64 machine with <br>
only 16GB or RAM.<br>
It can build llvm, but it can&#39;t build rust*, chromium, and some other <=

extra-large packages.<br>
Poudriere in QEMU can&#39;t build rust, and not even python312.<br>
I need rust-1.98.1.pkg for 15/aarch64 and 16/aarch64.<br>
The rust-1.98.1.pkg for 16 is ready in the builder but the builder is <br>
slow and it would take 5 more days to finish the batch.<br>

Questions:<br>
1. Is it possible to download built packages directly from builder servers?= <br></blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">= Have you tried this?</div><div dir=3D"auto"><br></div><div dir=3D"auto"><a = href=3D"https://github.com/freebsd/poudriere/blob/8b41a68f93647529950036c85= f73c7f90ad72a07/src/etc/poudriere.conf.sample#L386">https://github.com/free= bsd/poudriere/blob/8b41a68f93647529950036c85f73c7f90ad72a07/src/etc/poudrie= re.conf.sample#L386</a></div><div dir=3D"auto"><br></div><div dir=3D"auto">= <div class=3D"gmail_quote gmail_quote_container"><blockquote class=3D"gmail= _quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:= 1ex">
2. Is there a clean way to add such packages into Poudriere? Do I drop <br>
the package file into the packages/**/All folder?<br>



Thanks,<br>
Yuri<br>

</blockquote></div></div></div>

--00000000000044a7dd065bb35bea--


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports

On 9/17/26 13:09, Mark Millard wrote:

I should have noted:

) ALLOW_MAKE_JOBS=yes goes in a *poudriere.conf file.
(It is not from the ports environment: poudriere specific.)

) MAKE_JOBS_NUMBER=??? goes in a *make.conf instead.
(It is from the ports environment, not poudriere.)

FYI, based on my experience with 4 cores and 16 GiB, I've set my poudriere.conf with

PARALLEL_JOBS=1 # Build 1 port at a time
ALLOW_MAKE_JOBS=yes # Let each builder use multiple cores

If something like llvm or rust ends up getting built as a dependency on
this machine, and I don't allow make jobs, that build will take forever
and likely be 90% or more of the total build time. Even with 4 cores it
still often ends up being the majority of the total time.

With limited cores and RAM, building llvm/rust + anything else big at
the same time causes a lot of swapping, so PARALLEL_JOBS > 1 often
backfires. Ports that don't benefit from make jobs are usually quick
anyway (e.g. perl or python ports), so running them in parallel doesn't
help nearly as much as make jobs for the big ones.

On my 16 core + 64 GiB server, I set PARALLEL_JOBS to 2 or 3 and allow
limited make jobs (e.g. ALLOW_MAKE_JOBS=yes, MAKE_JOBS_NUMBER=8).

It would be nice if poudriere knew in advance how many cores and how
much RAM are needed for each individual port, and scheduled resources dynamically to maximize utilization at all times, but that's another topic.
--
Life is a game. Play hard. Play fair. Have fun.


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2