On Sun, Dec 08, 2024 at 04:53:58AM +0000, Sam James wrote:

I fear this sort of assumes we won't switch to monobuild any time soon.

I keep thinking [0] about how sustainable our current setup is:
* Fedora moved away from it for >=18 [1].
* As we saw with offload, it broke a few times in just a week. So it's
only Gentoo who cares about this configuration AFAIK.
* It's not working great if we're already not able to easily package
mlir, flang, or polly.

Violet attempted working on a merge before [2].

Hey! Thanks for bringing up my work on this. I've also been continuing
this work in an overlay which I use myself on some of my machines: https://codeberg.org/vimproved/llvm-overlay.

IIRC, Violet ended up running into issues with getting the CMake to
properly handle both shared and static libraries to install with the monobuild, so maybe LLVM's CMake is just broken in both directions and
we just have to pick whichever one is least-worst for us (which might be
the standalone builds as we do now)?

LLVM's CMake is definitely just rather broken in many ways, yeah. The
biggest issue I experienced when working on this was with LLD libraries. Currently, sys-devel/lld is built with -DBUILD_SHARED_LIBS=ON, which
builds some LLD libraries used by e.g. zig as shared libraries. However, there's no way to do this in monobuild since passing
-DBUILD_SHARED_LIBS=ON is not desirable. The solution I settled on was splitting a sys-libs/lld-libs package which builds LLD with -DBUILD_SHARED_LIBS=ON and then only installs the shared libraries.

As for sys-libs/llvm-libs, this package is a result of me trying to
figure out a way to have a statically linked system clang (for
performance purposes) while still having LLVM functional as a library.
After a lot of sifting through CMake spaghetti I settled on splitting
the LLVM monobuild into two packages: sys-devel/llvm and
sys-libs/llvm-libs. sys-devel/llvm installs the toolchain parts of LLVM (statically linked), and sys-libs/llvm-libs installs the library and
CMake parts of LLVM.

This overlay is just for my personal use, and I don't think the above
package split would be viable for ::gentoo however.

One more thing that wasn't mentioned here, LLVM monobuild would also potentially unblock PGO/BOLT optimizations for clang, so overall it
could have some pretty good speed improvements for system LLVM toolchain
users.

- Violet

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 2024-12-08 at 04:53 +0000, Sam James wrote:

I fear this sort of assumes we won't switch to monobuild any time soon.

I don't see one precluding the other. Categories are cheap. Package
moves not necessarily, but switching to monorepo will be complete pain
whether one more package move is involved or not.

I keep thinking [0] about how sustainable our current setup is:
* Fedora moved away from it for >=18 [1].
* As we saw with offload, it broke a few times in just a week. So it's
only Gentoo who cares about this configuration AFAIK.

It broke once, and only because the pull request merged preceded my
changes, and the author dealt with merge conflicts wrong.

That said, it's not like I didn't fix the monorepo build as well this
week, because it was broken from day one.

We're on our own either way.

* Build time

Build time for mono LLVM would increase as we're building more
components at least for some users.

But the time added by
building more components (see below) should be balanced out by ccache if
doing development and also, importantly, not needing to force on all
targets anymore (they keep growing).

I don't see how we would avoid forcing targets if *external* projects
(wasn't the bug about Rust originally?) can still be broken if you
change targets.

The cumulative time should be the same (although maybe a bit less
given the targets change) given that most people need the
same set of components because of mesa, firefox, or other things which
need libclang.

So you spend hours building LLVM and Clang. Then you spend hours
building everything again because one more packages needs LLD. Then
more hours because you've decided to try LLDB.

I've been rebuilding three LLVM versions recently because of cpp-httplib changing subslot multiple times recently. With the proposed change, I'd
be rebuilding everything instead.

In fact, I've already started considering splitting llvm-debuginfod.

At the moment, I fear us choosing the non-recommended path gives us very little, and causes a bunch of problems in return.

If you consider being able to have a really working LLVM package "very
little", so be it.

If you choose to go for monorepo, I'm stepping down, because
I definitely won't be able to deal with this shit without being able to
split it into smaller parts.

I don't like the idea that any minor patch (think of compiler-rt that
regularly needs to be updated for newer glibc) will require spending
hours rebuilding everything. In multiple LLVM versions. For every
single person, including all the people who don't build compiler-rt
at all.

I don't like the idea of not being able to run parts of test suite
without resorting to ugly hacks. I don't like the idea of spending
hours building everything before I'm even able to start running tests,
just to learn that LLVM is broken and there's no point in even starting
to build the rest. Or having the test suite fail after a few hours
because of some minor configuration issue (like the one we'd had with
libcxx, and I've hit that one three times), and having to start
everything over again.


And ccache is not a solution. It's a cheap hack, and a costly one. Maintaining a cache for this thing requires tons of wasted disk space.
And unless you go out of the way to reconfigure it, building 2-3 LLVM
versions will normally push all previous objects of the cache, so it
won't work for most of the people at all. Provided they go out of their
way to configure it in the first place.

In the end, LLVM is a project primarily maintained by people working for
shitty corporations that only care about being able to build their
shitty browser written in bad C++. It sucks we ended up having to
maintain it, but that's not our choice.

--
Best regards,
Michał Górny


-----BEGIN PGP SIGNATURE-----

iQFGBAABCgAwFiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAmdVnRgSHG1nb3JueUBn ZW50b28ub3JnAAoJEGOa2uIyniQOqcgIALLojaaEuifZx2S0DUV4HqUFUABLFP68 DXisMHSEBZ0yKJ32F7yjkLl2KnOJMqprqh4AdhyrISvk0M9+l8shZIyZ3lxGfnoU LsKlxaqr5XxK6AMeobnSM6ov/ot1hR+nNQBu2NqU+gk1WzNFkepp3g2wJQo0VEhO 4zDJNV6K+RBMWuCJZ1etJprUVCMHoCSKgbOTndwIWH9hq4awUoReAyee/17Akr31 54qmAHLyayKzNnTpM2qLtjBPBgao5GoVWRCkNnhM5oj5Q+FCFNlHyBr8FUQUkjuW KejIGNbqO7ilRsRNmvj3SY8J7aL1Y7v5t1frNrxOhjPSiCxx/yEGbdE=
=LE1m
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)