A question about --as-needed as an upstream developer who wants to
care about more than just Debian or Linux. Am I correct in assuming
this will work on any system using GNU binutils? And it doesn't
matter whether you are using gcc, clang, etc. What about other OS's
such as *BSD, MacOS, etc.?

I guess I could add an autoconf test to see if the linker barfs on
--as-needed, but I'm curious how much this actually matters in
practice.

Thanks,

- Ted

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, Jan 21, 2025 at 10:31:29AM -0500, Theodore Ts'o wrote:

A question about --as-needed as an upstream developer who wants to
care about more than just Debian or Linux. Am I correct in assuming
this will work on any system using GNU binutils? And it doesn't
matter whether you are using gcc, clang, etc. What about other OS's
such as *BSD, MacOS, etc.?

I often consult Gnulib for lore on this kind of thing. https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=m4/lib-ignore.m4 seems to have some useful hints, with comments for those not fluent in
m4.

This also seems to be a standalone file, so if your project is using
Autoconf but not Gnulib, then you should be able to copy it into your
project's local macro directory, add gl_IGNORE_UNUSED_LIBRARIES to configure.ac, and add $(IGNORE_UNUSED_LIBRARIES_CFLAGS) etc. to whatever *_LDFLAGS variable is appropriate.

--
Colin Watson (he/him) [cjwatson@debian.org]

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 21.01.25 22:01, Colin Watson wrote:

On Tue, Jan 21, 2025 at 10:31:29AM -0500, Theodore Ts'o wrote:

A question about --as-needed as an upstream developer who wants to
care about more than just Debian or Linux. Am I correct in assuming
this will work on any system using GNU binutils? And it doesn't
matter whether you are using gcc, clang, etc. What about other OS's
such as *BSD, MacOS, etc.?

--as-needed has been there forever. The --push-state/--pop-state options
are implemented in binutils 2014 (ld), and binutils 2016 (gold). I
didn't look at other linkers.

Using these in libtool still seems to be a problem, as libtool has it's
own idea about how to re-sort command line options and libraries.

I often consult Gnulib for lore on this kind of thing. https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob;f=m4/lib-ignore.m4 seems to have some useful hints, with comments for those not fluent in
m4.

This also seems to be a standalone file, so if your project is using
Autoconf but not Gnulib, then you should be able to copy it into your project's local macro directory, add gl_IGNORE_UNUSED_LIBRARIES to configure.ac, and add $(IGNORE_UNUSED_LIBRARIES_CFLAGS) etc. to whatever *_LDFLAGS variable is appropriate.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On 1/22/25 00:31, Theodore Ts'o wrote:

I guess I could add an autoconf test to see if the linker barfs on --as-needed, but I'm curious how much this actually matters in
practice.

From an upstream perspective, I think the best approach is

AC_CHECK_LIB([atomic],[main])

That creates a few unnecessary dependencies, but it should work pretty
much everywhere.

If Debian maintainers feel annoyed by warnings about unnecessary
dependencies, they are welcome to replace this with a hardcoded

LDFLAGS="${LDFLAGS} -Wl,--push-state,--as-needed,-latomic,--pop-state"

because unlike upstream, Debian can assume reasonably recent GNU tools.

Someone GNU adjacent might want to build something more sophisticated
for the autoconf-archive -- but I'd draw the line at

0. if the library does not exist, assume it's not needed and explicitly
say so in the build log
1. if --push-state/--pop-state are supported, wrap them around --as-needed
2. if --as-needed is supported, but no state stack, use --no-as-needed afterwards, and if any other macro wants to use --as-needed, they need
to add their own
3. if neither is supported, unconditionally link

Like libm, what is actually needed at link time depends on inlining,
which is affected by constant propagation, so it is impossible to build
100% reliable tests.

Simon

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On 1/17/25 17:45, Alexandre Rossi wrote:

Some upstreams have gone through build system tests to assess whether
to build with libatomic or not. Adding -latomic is simpler. I wonder if
the build system configure stage test is overkill.

Yes and no. A lot of the tests are just broken, like "compile and link a program incrementing an atomic integer, try with no extra libraries and
with libatomic" -- in principle you'd need to repeat this with all data
types used, then you'd get a reliable result.

It's similar to how libm is used -- if you use fsin(), this may be a
single assembler instruction, or a call to a function that is IEEE754 conformant and sets errno, and that implementation detail determines
whether you want to link against libm or not, but if you test whether
fsin() requires -lm, that doesn't tell you whether fabs() requires -lm.

Likewise, 4-byte atomics being inlined doesn't tell you if 1-byte or
16-byte atomics are also inlined, or whether there is special handling
for signed or unsigned integers.

Within Debian, always linking with --as-needed is a simple fix, but it
depends on using a linker that understands --as-needed, so it's not
portable enough to be submitted upstream.

In my own packages, I check if libatomic exists, and if it does, I unconditionally link if I use any atomics. I also check if the linker
accepts --push-flags, if it does I generate a -Wl,--push-flags,--as-needed,-latomic,--pop-flags sequence, if not, the unconditional link will generate dpkg-shlibdeps warnings about
unnecessary linking on amd64, but that's better than failing on other platforms.

Simon

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-17 19:43 +0900, Simon Richter wrote:

In my own packages, I check if libatomic exists, and if it does, I unconditionally link if I use any atomics. I also check if the linker
accepts --push-flags, if it does I generate a -Wl,--push-flags,--as-needed,-latomic,--pop-flags sequence, if not, the unconditional link will generate dpkg-shlibdeps warnings about unnecessary linking on amd64, but that's better than failing on other platforms.

Can you explain why this:
-Wl,--push-flags,--as-needed,-latomic,--pop-flags
is better than
-as-needed,-latomic
I thought --as-needed on its own is sufficicent to avoid dpkg-shlibdeps warnings
about unnecessary linking (because it only links the things that are needed), so
I don't understand the need for the psuh/pop sequence (I must admit that I never
knew that existed until your message recently).

cheers

Wookey
--
Principal hats: Debian, Wookware
http://wookware.org/

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEER4nvI8Pe/wVWh5yq+4YyUahvnkcFAmeKeAAACgkQ+4YyUahv nkfv7RAAlvsPIID0wjA8p+JvZSv132lj4biwzK94zvHlw9fqqeaBieFql2VeP3Ru S4VlJ5ToiOH6SPnGB5y1jnRVwcSlYVwFMoqgKLrdEul2IcH1nVTd/94igwXSUH8j JR+WfSwkqVDjeYj0TH4SD3jwG5GH89IrXaoOwSDyOhzfkLlS5CDj031GNyMwq+8u a/LLIXZHGuOJmQ1lgQtRD6bS+8F+w7eYCLz97JA8QTvsJW4BLVMllJKhsOsaRzFI T/++oyexAwyoM74hWSOWn/83Mff0G29MofGswUy57qjYAxfhYYkmKBweAdtTLyqA 5aggOiOLd6aQ5V/71SNgBcY4PkzXoExlLDE3G9Qr1vaQt8t/BJ6EeNOAPhdcXBJz oeZzpAprdxerhBVvvluUrYIAogUEIz92kcwVpFWS9y5pdhV3xfJicpyLKN7b/cx1 hatKL0TxF2fElEKfOaj8eRISjgSpMdJQhC7qxXqsUWWTjk2ZDInSRpt2NpvS3Dvj xGvKEVOploxClQ+aXOrshK2xRBV1JDnBh2hZ5r51itFqNWvfHBYKv+c1nfZ4rKuI oLS6RpUIbSMMLn9P2AhxDCBaXOgHMsljtw3gh+MssSOkL4SKjHzQ2RxaevuSOZGS fv7JfatNPF5Uj75+rh7YQyaJhTG0EB5niePBma8Lv2Gljvowvyo=
=7Mvw
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 17 Jan 2025 at 15:32:22 +0000, Wookey wrote:

Can you explain why this:
-Wl,--push-flags,--as-needed,-latomic,--pop-flags
is better than
-as-needed,-latomic
I thought --as-needed on its own is sufficicent to avoid dpkg-shlibdeps warnings
about unnecessary linking (because it only links the things that are needed), so
I don't understand the need for the psuh/pop sequence (I must admit that I never
knew that existed until your message recently).

Two things, depending which difference between your two command lines
you are thinking about:

1. -Wl,--as-needed normally sets global state for the entire linker
invocation. If the package you're building is one that has a
functional requirement to *not* use -Wl,--as-needed for other library
dependencies, the push/pop guards ensure that the -Wl,--as-needed
only affects how libatomic is linked, without having the side-effects
of linking every other library dependency (not just libatomic!) in
"as needed" mode.

For example GLib explicitly pulls in libpthread, which is (or at least
was in the past) functionally necessary: if that wasn't done,
dlopen()ing modules that indirectly depend on libpthread would not
be safe, because internal locks in glibc were already initialized
in a non-thread-safe way before libpthread was in memory, and cannot
(or at least could not in 2009) be re-initialized with the thread-safe
equivalent after libpthread arrived. (Reference:
https://mail.gnome.org/archives/gtk-devel-list/2009-November/msg00096.html)

2. -Wl is necessary to tell the compiler driver (gcc or clang)
"it's OK that you don't understand this, just pass it through to the
linker". --as-needed is a ld option, not a gcc option; but $LDFLAGS
are passed to gcc, not ld.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

Quoting Simon Richter (2025-01-17 11:43:39)

In my own packages, I check if libatomic exists, and if it does, I unconditionally link if I use any atomics. I also check if the linker accepts --push-flags, if it does I generate a -Wl,--push-flags,--as-needed,-latomic,--pop-flags sequence, if not, the unconditional link will generate dpkg-shlibdeps warnings about unnecessary linking on amd64, but that's better than failing on other platforms.

thank you for your help! I now patched vcmi with snippets like this:

if (CMAKE_LIBRARY_ARCHITECTURE STREQUAL "arm-linux-gnueabi")
target_link_options(vcmi PRIVATE "-Wl,--push-flags,--as-needed,-latomic,--pop-flags")
endif()

I understand from the other mails that this architecture-check is not even needed and that --as-needed will let the linker do the right thing. I still like to special-case the only arch where this seems to be needed. I gave up on hoping that onetbb fixes it as neither upstream nor the Debian bug show any activity, so I'm patching my own package instead.

But then I get the complaint:

/usr/bin/ld: unrecognized option '--push-flags'

You probably meant --push-state and --pop-state instead?

Thanks!

cheers, josch
--==============u78752677170248125=MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmepy28ACgkQ8sulx4+9 g+E3eBAAn3kg9g6S5tvN5jWNZhtm84CxnHkTEtC/s7t1w1nPlNjq0AZE0AV4UIKn GTe1MK4ezf2eoLBiUBWRznoOb/J3A0Hl/bK/ijH2D0Og0m20CGIDa9CGivYLFEpY YB37bdmMdgxeA8Z5TkXrCv+x9/dpn6462AYNQ40c3xAoL2Lop8/Xe8tQY0i5LOA8 xJmwDR/0la6IFo48xtNkI9/Fc8/fLCbATK8WhDTp07FMA1kSC7SyhPKgddqPXTRf 92tvQ4PIBvpLF42wuyDMNeax09vZAEQTexTqtzkVCaf7k7p2q6AZYCk+kpGCiabp LcPDer/pAqUlI8h/gjN98U/lfTnoIL1Nkb/Yx9qeX3+HEScpJi/zoFPnH1+s/teB ft4IeSFJVt9cFJtU5RgYAC6cAhIrXPmTjhK0JT8Q/hiIok6MFi8jOQTixY4DO8zc RnQg0a7j/euwXW5XHT0UhPtpxYOtkd8hbBy17T4BaTK6k6g/80Z0f9P8mYwEM+PZ gZdt5gqsHKTDY8HyXoUBY5kZuVQqx1SfT0q/njhkWfAYsYfZhTwYTiHcglxkwLND Be4hxRKRoBx5Dqj3loLJwYP8Mz2/P5KJYSA/IQ53fo9NVU0v8bSN35dK/TnMysYN J28RWpyBFLe8IFb+ZJDD+xVzlWRA80ea7B34+9XTGSmnikPEiuQ=
=rFTX
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On 2/10/25 18:48, Johannes Schauer Marin Rodrigues wrote:

I understand from the other mails that this architecture-check is not even needed and that --as-needed will let the linker do the right thing. I still like to special-case the only arch where this seems to be needed.

It's not needed on RISC-V? Whoa.

I gave up on
hoping that onetbb fixes it as neither upstream nor the Debian bug show any activity, so I'm patching my own package instead.

FWIW if the offending code is inlined into your package, then there
isn't much they can do to fix it.

/usr/bin/ld: unrecognized option '--push-flags'

You probably meant --push-state and --pop-state instead?

Yes, indeed.

Simon

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Quoting Johannes Schauer Marin Rodrigues (2025-02-10 10:48:35)

Quoting Simon Richter (2025-01-17 11:43:39)

In my own packages, I check if libatomic exists, and if it does, I unconditionally link if I use any atomics. I also check if the linker accepts
--push-flags, if it does I generate a -Wl,--push-flags,--as-needed,-latomic,--pop-flags sequence, if not, the unconditional link will generate dpkg-shlibdeps warnings about unnecessary linking on amd64, but that's better than failing on other platforms.

thank you for your help! I now patched vcmi with snippets like this:

if (CMAKE_LIBRARY_ARCHITECTURE STREQUAL "arm-linux-gnueabi")
target_link_options(vcmi PRIVATE "-Wl,--push-flags,--as-needed,-latomic,--pop-flags")
endif()

I understand from the other mails that this architecture-check is not even needed and that --as-needed will let the linker do the right thing. I still like to special-case the only arch where this seems to be needed. I gave up on
hoping that onetbb fixes it as neither upstream nor the Debian bug show any activity, so I'm patching my own package instead.

But then I get the complaint:

/usr/bin/ld: unrecognized option '--push-flags'

You probably meant --push-state and --pop-state instead?

probably related to how to correctly link against atomic with CMake I have another follow-up question. Adding the above target_link_options() to the relevant shared libraries indeed does the trick. But for my package vcmi, I also seem to need it for an executable. Unfortunately, when I apply the same recipe, I get this:

[100%] Linking CXX executable ../bin/vcmiclient
cd /build/reproducible-path/vcmi-1.6.5+dfsg/obj-arm-linux-gnueabi/clientapp && /usr/bin/cmake -E cmake_link_script CMakeFiles/vcmiclient.dir/link.txt --verbose=1
/usr/bin/c++ -g -O2 -ffile-prefix-map=/build/reproducible-path/vcmi-1.6.5+dfsg=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2
-Wall -Wextra -Wpointer-arith -Wuninitialized -Wmismatched-tags -Wno-unused-parameter -Wno-switch -Wno-reorder -Wno-sign-compare -Wno-varargs -Wl,-z,relro -Wl,-z,now -Wl,--push-state,--as-needed,-latomic,--pop-state -Wl,--dependency-file=CMakeFiles/
vcmiclient.dir/link.d CMakeFiles/vcmiclient.dir/StdInc.cpp.o CMakeFiles/vcmiclient.dir/EntryPoint.cpp.o -o ../bin/vcmiclient -Wl,-rpath,"\$ORIGIN::::::::::::::::::::::::" ../bin/libvcmiclientcommon.a ../bin/libvcmiservercommon.a ../bin/libvcmi.so /usr/
lib/arm-linux-gnueabi/libminizip.so /usr/lib/arm-linux-gnueabi/libz.so /usr/lib/arm-linux-gnueabi/libtbb.so.12.14 -ldl -lrt /usr/lib/arm-linux-gnueabi/libboost_filesystem.so.1.83.0 /usr/lib/arm-linux-gnueabi/libboost_program_options.so.1.83.0 /usr/lib/
arm-linux-gnueabi/libboost_locale.so.1.83.0 /usr/lib/arm-linux-gnueabi/libboost_thread.so.1.83.0 /usr/lib/arm-linux-gnueabi/libboost_atomic.so.1.83.0 /usr/lib/arm-linux-gnueabi/libboost_chrono.so.1.83.0 /usr/lib/arm-linux-gnueabi/libboost_date_time.so.1.
83.0 /usr/lib/arm-linux-gnueabi/libSDL2_image.so /usr/lib/arm-linux-gnueabi/libSDL2_mixer.so /usr/lib/arm-linux-gnueabi/libSDL2_ttf.so /usr/lib/arm-linux-gnueabi/libSDL2.so /usr/lib/arm-linux-gnueabi/libavutil.so /usr/lib/arm-linux-gnueabi/libswscale.so /
usr/lib/arm-linux-gnueabi/libavformat.so /usr/lib/arm-linux-gnueabi/libavcodec.so /usr/lib/arm-linux-gnueabi/libswresample.so
/usr/bin/ld: ../bin/libvcmiclientcommon.a(SDLImageScaler.cpp.o): undefined reference to symbol '__atomic_fetch_add_8@@LIBATOMIC_1.0'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabi/14/libatomic.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

I suspect that the problem is the order in which the -latomic is added to the linker flags? Because if instead of target_link_options() with push and pop-state I just use

target_link_libraries(vcmiclient PRIVATE atomic)

Then that will add a -latomic right after ../bin/libvcmiclientcommon.a and then the build succeeds. Here is the relevant CMakeLists.txt:

https://sources.debian.org/src/vcmi/1.6.5%2Bdfsg-1/clientapp/CMakeLists.txt/#L38

So what is the canonical way to achieve the desired result with CMake? Will I instead have to globally set LDFLAGS="-Wl,--as-needed"?

Thanks!

cheers, josch
--==============b53451177148606974=MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmesJccACgkQ8sulx4+9 g+HA4w//XdL+nizzaxeIw/OBUbocI4z3Lgh97rwYJ43I9J6nQu0cY+j3u5FPuap2 oXF8nlyz5VXWtj5WHi8s5+/Mpp5tf+wxGHPASb0k4uOnnl8F9s/p5OHA9WZW4slx Z1ZCHByA2HHoDSEjzuZLoXnnce1+e6c6raN3E1EhFJ3vH8rs0DhyU6DS2b+5BN6e 77H7CGaSdfJLqWlXo4CdoR0Z1uBd5FPpztMH5wcQLYSFSxU5Hh2AqaOf+eSv8T+q qBBfw9ojc8NXX92NHgQ2sUv2tVsF0jdQYCCJS4Zh1ZNvT6v2FyONs4o+5hZUFrok yGbNVEb9i5GHx/Y27gylrGrKQ5u2NRs9I7qbl23Bc2LBMQ9VSitS4PjtA4x4Iyhh O4uRzFQV41rUouzwTQlrZgcx3xsMMamgBTYjJULo5QRaFI3xqXmet17YvfoDOQqt oi4BVjhUwK9FXsfiINUgYnnJCK+fGAF8908H36h3PxN+UUvc1y1CIjqpw57EoeMQ pg7Rco9KU54eFP/EbnCgLH3A7xNODusrjx/SW8qO3cehBzh9hAaDHNMO4U2zpo2Y YlfjeuezUw0op81l6sqkQ1Uyr5XHFLJkm4DKsB3oFt2+Z0mfrQPAxm51N9PsGL8G XgYSPUoZ1pnK9mmQ2vz6tgKp34zZpJOOO7C8B+wtW71cQVB2/Uc=
=F1z2
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On 2/12/25 13:38, Johannes Schauer Marin Rodrigues wrote:

["DSO missing from command line"]

I suspect that the problem is the order in which the -latomic is added to the linker flags?

Yes.

Because if instead of target_link_options() with push and
pop-state I just use

target_link_libraries(vcmiclient PRIVATE atomic)

Then that will add a -latomic right after ../bin/libvcmiclientcommon.a and then
the build succeeds.

Yes, that is the correct approach -- that target is dependent on libatomic.

So what is the canonical way to achieve the desired result with CMake? Will I instead have to globally set LDFLAGS="-Wl,--as-needed"?

On Debian, that is default already, which is very annoying when working
around #1002981 -- I need to add

[[maybe_unused]]
volatile int (*hack)(Display, char const *) = XMissingExtension;

to force a symbol reference from my code to libXext and avoid unloading
libXext before my program exits.

The --push-state logic is more useful for sending upstream.

Simon

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)