As subject. Why does devel/electron41 show as blacklisted?
https://pkg-status.freebsd.org/package19/build.html?mastername=150amd64-default-build-as-user&build=bc5f8cc82e1a
All the electron ports command very huge resources to build.
Like > 24h on a beefy machine.
Therefore only one will currently build and we hope that
all the ports that depend on some version of electron try to converge
on the one that is being built by the ports cluster.
On Mon, May 25, 2026 at 03:24:25PM +0200, Kurt Jaeger wrote:
All the electron ports command very huge resources to build.
Like > 24h on a beefy machine.
yeah I've found that too.
Is there a list available of what falls into this category?
I mean, others which aren't electron based which also
take huge resources to build.
Typical desktop self-built update takes 10 days on 4 real cores machine. That is the reality today :/Therefore only one will currently build and we hope that
all the ports that depend on some version of electron try to converge
on the one that is being built by the ports cluster.
electron41 here is being built as dependency of signal-desktop.
The electron port wasn't manually configured to do this on
my poudriere instance.
--
Is there a list available of what falls into this category?
I mean, others which aren't electron based which also
take huge resources to build.
Therefore only one will currently build and we hope that
all the ports that depend on some version of electron try to converge
on the one that is being built by the ports cluster.
electron41 here is being built as dependency of signal-desktop.
The electron port wasn't manually configured to do this on
my poudriere instance.
On Mon, May 25, 2026 at 03:24:25PM +0200, Kurt Jaeger wrote:
All the electron ports command very huge resources to build.
Like > 24h on a beefy machine.
yeah I've found that too.
Is there a list available of what falls into this category?
I mean, others which aren't electron based which also
take huge resources to build.
--Therefore only one will currently build and we hope that
all the ports that depend on some version of electron try to converge
on the one that is being built by the ports cluster.
electron41 here is being built as dependency of signal-desktop.
The electron port wasn't manually configured to do this on
my poudriere instance.
Hi!
Is there a list available of what falls into this category?
The topic is discussed here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270565
I mean, others which aren't electron based which also
take huge resources to build.
Yes. It's complicated.
I mean, others which aren't electron based which also
take huge resources to build.
Yes. It's complicated.
Could compute be distributed?
Desktop essentials have grown huge :(
Hi!
I mean, others which aren't electron based which also
take huge resources to build.
Yes. It's complicated.
Could compute be distributed?
Desktop essentials have grown huge :(
The ports tree is a dependency tree. To target the fastest way for
upgrades would require to rebuild if a dependency changes
and to optimise the longest path, compile-time-wise.
Finding that longest path would be interesting.
Some elements in that path might have problems with parallel
compilation (MAKE_JOBS_UNSAFE option). Fixing those elements
might help.
Because this longest path is not subject to parallelization, I'm
not sure it's easily achievable.
It is subject to CPU takt rate optimization, therefore using faster
CPUs might help as well. And large TMPFS.
The ports tree is a dependency tree. To target the fastest way for
upgrades would require to rebuild if a dependency changes
and to optimise the longest path, compile-time-wise.
Finding that longest path would be interesting.
Some elements in that path might have problems with parallel
compilation (MAKE_JOBS_UNSAFE option). Fixing those elements
might help.
Because this longest path is not subject to parallelization, I'm
not sure it's easily achievable.
It is subject to CPU takt rate optimization, therefore using faster
CPUs might help as well. And large TMPFS.
The ports tree is a dependency tree. To target the fastest way for
upgrades would require to rebuild if a dependency changes
and to optimise the longest path, compile-time-wise.
I don't think I articulated what I meant thoroughly.
By distributed I mean something like folding@home where a user
would get a slice of work and process/upload it.
Hi!
The ports tree is a dependency tree=2E To target the fastest way for
upgrades would require to rebuild if a dependency changes
and to optimise the longest path, compile-time-wise=2E
I don't think I articulated what I meant thoroughly=2E
By distributed I mean something like folding@home where a user
would get a slice of work and process/upload it=2E
Again, it's a dependency tree=2E You can not parallelize more=20
than a certain number, because nodes would have to wait for other nodes
to finish before they can work=2E So one has to reduce
the dependencies and shorten the longest path=2E
--=20
pi@FreeBSD=2Eorg +49 171 3101372 Now what ?
I do believe they're talking about outsourcing the build process and distributing amongst resources. Like a hive or a chain where distributed systems would be used to build pieces (of the software) and collaborate back (which does have potential to be a great resource.) But, the Poudriere system does not support this currently. Perhaps in the future?
Hi!
Again, it's a dependency tree. You can not parallelize more
than a certain number, because nodes would have to wait for other nodes
to finish before they can work. So one has to reduce
the dependencies and shorten the longest path.
I do not see in void's wording anything indicating that the unit of work
would be a builder run of one port-package-build.
Yes. But if you have 10000 cores of the same GHz Rating as
one fast box with only, let's say, 128 cores.
If you want to measure from the first to the last package,
the way to shorten the total build time is to analyse and
optimize the longest path in the dependency tree.
. . .
Hi!
I do believe they're talking about outsourcing the build process and distributing amongst resources. Like a hive or a chain where distributed systems would be used to build pieces (of the software) and collaborate back (which does have potential to be a great resource.) But, the Poudriere system does not support this currently. Perhaps in the future?
Again: The critical path for a rebuild of the ports tree is
the longest compile-time path in the dependency tree.
If you throw more CPUs at the problem, the longest path will
not become shorter. Only if you use CPUs with higher GHz rate.
--
pi@FreeBSD.org +49 171 3101372 Now what ?
On Wed, 27 May 2026 18:01:51 +0200
Kurt Jaeger <pi@freebsd.org> wrote:
Hi!
I do believe they're talking about outsourcing the build process and distributing amongst resources. Like a hive or a chain where distributed systems would be used to build pieces (of the software) and collaborate back (which does have potential to be a great resource.) But, the Poudriere system does not support this currently. Perhaps in the future?
Again: The critical path for a rebuild of the ports tree is
the longest compile-time path in the dependency tree.
If you throw more CPUs at the problem, the longest path will
not become shorter. Only if you use CPUs with higher GHz rate.
--
pi@FreeBSD.org-a-a-a-a-a-a-a-a +49 171 3101372-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a Now what ?
What I read from this thread is a bit different.
Aren't others talking about "Can't be a build for single huge port
to be built distributed?" rather thand "Can't we build more ports
at once?"?
Hi!
Well, I can confirm that electron can use 4 real cores, the build than takes around 23 hours! Has anybody experimented with larger number of cores? 32, for example?
On Fri, May 29, 2026 at 09:20:57PM +0200, Bugs Beastie wrote:Just finished one build:
Hi!
Well, I can confirm that electron can use 4 real cores, the build than takes around 23 hours! Has anybody experimented with larger number of cores? 32, for example?
I'll try a from-scratch build on a dual Xenon E5-2690 with 384GB RAM and report back here.
The SHT is turned off so there's 16 real cores.
--
On Sat, May 30, 2026 at 05:26:37PM +0200, Bugs Beastie wrote:
Just finished one build:
[4D:08:25:51] [01] [21:03:31] Finished-a-a devel/electron39 | electron39-39.8.10_1: Success
21 hours for electron39, this is on 4x3.3GHz core i5!
[electron41 failed in a strange way in fetch/checksum phase, no data for it now]
[07:51:54] [01] [06:04:22] Finished-a-a devel/electron41 | electron41-41.7.1: Success
Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz / 384GB RAM hw.ncpu=16 SHT=off
Probably this is the most reasonable direct way to go, to introduce "desktop builder(s)"! It will get all "small" packages already built from regular builders. It will crunch heavy desktop ports with only single poudriere builder, but with all available cores given to MAKE_JOBS!
Note that such setup is almost opposite to "regular builders" that have a small amount of cores per poudriere builder, but large number of poudriere builders!
Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz / 384GB RAM hw.ncpu=16 SHT=offIf it scales good further, 96 cores will manage electron in one hour! :)
On Sat, May 30, 2026 at 05:26:37PM +0200, Bugs Beastie wrote:
Just finished one build:
[4D:08:25:51] [01] [21:03:31] Finished-a-a devel/electron39 |
electron39-39.8.10_1: Success
21 hours for electron39, this is on 4x3.3GHz core i5!
[electron41 failed in a strange way in fetch/checksum phase, no data
for it now]
[07:51:54] [01] [06:04:22] Finished-a-a devel/electron41 | electron41-41.7.1: Success
This is a from-scratch build. In other words the poudriere builder was cleaned
with poudriere pkgclean -A -j jailname.
The build was started with 'doas poudriere bulk -j 151amd64 -J16 devel/electron4'
System is
Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz / 384GB RAM hw.ncpu=16 SHT=off
ccache4 is in use.
-a Cleaning for electron42-42.3.3build of devel/electron42 | electron42-42.3.3 ended at Sun Jun 14
=20Yes, it will often be competing with multiple llvm, gcc, and the rust =
Do you allow a bunch of other port package builds to potentially be
running in parallel?
=20
June 24, 2026 at 10:03 PM, "Mark Millard" <marklmi@yahoo.com <mailto:marklmi@yahoo.com? to=%22Mark%20Millard%22%20%3Cmarklmi%40yahoo.com%3E>> wrote:
Do you allow a bunch of other port package builds to potentially be
running in parallel?
Yes, it will often be competing with multiple llvm, gcc, and the rust
port. But it has never taken anywhere near as long as it does on the
build cluster.
Is it because I have 250GB of ccache on an NVME? I wouldn't expect it to
have such a significant impact but considering the size of the codebase perhaps it does.
Yes, it will often be competing with multiple llvm, gcc, and the rustPoudriere's scheduling is lousy (though it's mostly not its own fault).
port. But it has never taken anywhere near as long as it does on the
build cluster.
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 70 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 03:26:58 |
| Calls: | 949 |
| Calls today: | 1 |
| Files: | 1,325 |
| Messages: | 281,242 |