• Re: Proposed MBF: packages that FTBFS with make --shuffle

    From Adrian Bunk@21:1/5 to Lucas Nussbaum on Wed May 14 13:10:01 2025
    On Tue, May 06, 2025 at 08:48:29AM +0200, Lucas Nussbaum wrote:
    On 05/05/25 at 22:14 +0200, Santiago Vila wrote:
    In some cases, the bug is already known, because debian/rules
    has --max-parallel=1. Example: The alpine package.

    (I wonder how much feasible would be to skip those packages)

    The alpine package is indeed a good example of a package that makes
    extensive use of the sequentiality of 'make', and that is going to be
    hard to adjust to switch to parallel building or arbitrary orders.

    However I still think that there's value in filing bugs for such
    packages, because --shuffle=reverse makes it much easier to debug such issues: instead of trying a parallel build and getting a subtlely
    different race conditions at each run, you get a reproducible ordering
    that exhibits one issue that you can debug, and then move on to the next issue.

    Also it's not trivial to distinguish between packages that do not build
    in parallel on purpose, vs those that just happen not to build in
    parallel (yet).

    What is a maintainer supposed to do when the package already does
    "dh --no-parallel" and the upstream Makefiles are basically unfixable?
    Just close the bug?
    Strip "--shuffle" in debian/rules?

    How many of the packages that break with "make --shuffle" are currently
    doing parallel building?
    I am asking since these might be RC bugs for trixie.

    Lucas

    cu
    Adrian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Santiago Vila@21:1/5 to All on Wed May 14 19:00:01 2025
    El 14/5/25 a las 12:50, Adrian Bunk escribió:
    What is a maintainer supposed to do when the package already does
    "dh --no-parallel" and the upstream Makefiles are basically unfixable?

    Whether a Makefile is unfixable or not is subjective. Everything depends
    on the amount of time that one is willing to spend. It may be the case
    that a Makefile looks unfixable to you, and at the same time it
    may be a perfectly legitimate bug to be reported upstream.

    So, at the very minimum, I think maintainers are supposed to forward
    those bugs upstream.

    [...]

    How many of the packages that break with "make --shuffle" are currently
    doing parallel building?
    I am asking since these might be RC bugs for trixie.

    I believed (regarding build failures) that you only cared about RC-ness
    when the failure happens in the buildds. Have you changed your mind?

    Thanks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adrian Bunk@21:1/5 to Santiago Vila on Wed May 14 19:50:01 2025
    On Wed, May 14, 2025 at 06:58:22PM +0200, Santiago Vila wrote:
    El 14/5/25 a las 12:50, Adrian Bunk escribió:
    ...
    How many of the packages that break with "make --shuffle" are currently doing parallel building?
    I am asking since these might be RC bugs for trixie.

    I believed (regarding build failures) that you only cared about RC-ness
    when the failure happens in the buildds. Have you changed your mind?

    Exotic setups like single-cpu or swap-starved are not something that
    could happen on release architecture buildds, I do consider it harmful
    trying to force maintainers to waste time on supporting unsuitable build environments through RC bugs.

    Parallel build issues in packages that do claim to support parallel
    building are race conditions that do sometimes fail on the buildds.

    Like until recently no release architecture buildd was building with
    parallel > 4, but now parallel=8 is common. This is something where
    the first DSA/pu upload in trixie might fail when a race condition
    became more likely due to that.

    Thanks.

    cu
    Adrian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Santiago Vila@21:1/5 to All on Wed May 14 20:30:01 2025
    El 14/5/25 a las 19:27, Adrian Bunk escribió:
    On Wed, May 14, 2025 at 06:58:22PM +0200, Santiago Vila wrote:
    El 14/5/25 a las 12:50, Adrian Bunk escribió:
    ...
    How many of the packages that break with "make --shuffle" are currently
    doing parallel building?
    I am asking since these might be RC bugs for trixie.

    I believed (regarding build failures) that you only cared about RC-ness
    when the failure happens in the buildds. Have you changed your mind?

    Exotic setups like single-cpu or swap-starved are not something that
    could happen on release architecture buildds, I do consider it harmful
    trying to force maintainers to waste time on supporting unsuitable build environments through RC bugs.

    Ok, I see, you have not changed your mind... You keep ranting me about the work I do at every opportunity, and you are still campaigning against single-cpu systems without a good reason. FYI: As a general rule, I'm currently not reporting those as RC, unless I provide a patch, in which case it would be tasteless for the maintainer to reject it considering that it's a violation
    of a must directive in policy.

    I reject categorically your idea that single-cpu is exotic or unsuitable
    to build packages. The only really required thing to build packages is
    enough memory and enough disk.

    Single-CPU systems are ubiquitous in the cloud. I was a system admin in
    a small startup some time ago. Everything was in the cloud, and one of my duties was naturally to reduce the IT bill if possible, so I used single-cpu systems extensively, as they are almost always cheaper.

    Let me quote the Social Contract:

    We will support the needs of our users for operation in many
    different kinds of computing environments.

    So please stop your continuous harassing against my work. I'll continue
    to report every bug which as a user I would not like to find in a stable release, of course trying to follow the guidelines of release managers regarding severities and the like, but sorry, I will also continue to
    ignore your CPUist ideas about what is exotic and what is not when
    I decide *what* to report or *how* I do archive rebuilds.

    Please do not Cc:me anymore in this thread.

    Thanks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henrik Ahlgren@21:1/5 to Santiago Vila on Thu May 15 12:00:01 2025
    Santiago Vila <sanvila@debian.org> writes:

    Single-CPU systems are ubiquitous in the cloud. I was a system admin in
    a small startup some time ago. Everything was in the cloud, and one of my duties was naturally to reduce the IT bill if possible, so I used single-cpu systems extensively, as they are almost always cheaper.

    What about virtual machine instances with an odd number of CPU cores
    (larger than one)? I recall encountering some peculiar bugs with
    three vCPU machines years ago.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Cyril Brulebois@21:1/5 to All on Sat May 17 01:20:01 2025
    Soren Stoutner <soren@debian.org> (2025-05-05):
    Filing these bug reports sounds like a good idea to me. I don’t see
    any reason to wait as these will be severity:minor, so they won’t
    interfere with the trixie release.

    Filing now can trigger uploads to fix those minor bugs, meaning packages
    that absolutely do not meet requirements for freeze exceptions end up in unstable. Depending on circumstances, they might make migrations of other packages more complicated than they need to be.

    (I'll refrain from naming a particular example, but I'm aware of one of
    those without even searching for it, it just happened to show up on one of
    my specific “important for the release” radars…)


    Cheers,
    --
    Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
    D-I release manager -- Release team member -- Freelance Consultant

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEtg6/KYRFPHDXTPR4/5FK8MKzVSAFAmgnxqsACgkQ/5FK8MKz VSDuVA//batMldkRwUUNY+4y+5Qv2w4X3yGN87vvh2jU8G2tDu4wRInOmRksdDGD K/gRWfcb2XO1dpS3iGYYNAodTWTVfXWMy9q3iJw8xT5LvSMzROkUDX6LM1VUz48J fqu4u2uvG3ouQ2msc6VdUxNi/ctJogFNzRnHb+5PmMANWMEBRkzUE+Pz7oiZzERL a7rggagoIAdTlMqGsJwsSdcXNyUdVxyNeIGGHJPAaQfO0cMFIEFrnsgGwQvVD785 ZXA17Cn8TmLtC5RA9vFa1mVJtESWBgcuGj0kCylHmnyhgQdS7gSwxAYNXkWsikrs RCwwxIDfQmONZlhGNJMcb8mjoayZ+uhlNH8yx95NbLWsXvaU3/PePUD0oODmXGCL hHEoXMjAYZusZ2W4llUiGVQHGR3ATj5i0s7a6MiUwZ3tVjeLzK1FVDjlXcQQ26F0 uLM9dXbjQSrkMI9BGCfmWWBeLgcV0miiNc4RXVcXuIElfWOU5R2N+Qx6NtTNG3UY w2oIraenmji5XLa1irhfRbyg15qjykfs+m+SM9Z/AsmzF33kovqriwG8lTVmN8b0 VqKq4sRiNGBpa+7tn2A0dVQ60CujBmO2gzmCezdMRb37Xx+1gEO6uXsk1bqOqn1y HmgzfQQtl+D84zEWOGt19QnQts2+NAk+U2ETI+uzTZga/wacrEI=
    =GdkL
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    *
  • From Cyril Brulebois@21:1/5 to All on Sat May 17 03:00:01 2025
    Santiago Vila <sanvila@debian.org> (2025-05-17):
    El 17/5/25 a las 1:13, Cyril Brulebois escribió:
    Soren Stoutner <soren@debian.org> (2025-05-05):
    Filing these bug reports sounds like a good idea to me. I don’t see any reason to wait as these will be severity:minor, so they won’t interfere with the trixie release.

    Filing now can trigger uploads to fix those minor bugs, meaning packages that absolutely do not meet requirements for freeze exceptions end up in unstable. Depending on circumstances, they might make migrations of other packages more complicated than they need to be.

    (I'll refrain from naming a particular example, but I'm aware of one of those without even searching for it, it just happened to show up on one of my specific “important for the release” radars…)

    Hi. Discussing about the right time to report those bugs does not make
    much sense anymore, because Lucas already reported them :-)

    Given I mentioned spotting a resulting upload, it should be pretty clear
    that I do know that has happened.

    It seemed important to me to reply to the “I don't see any reason to
    wait” part (which I quoted, and kept above just in case), so that people might take that in consideration next time they want to MBF that late in
    the freeze.

    Now we can only expect maintainers to act responsibly, and as you
    point out, be particularly careful about not affecting other packages.

    The example I was alluding to builds a udeb, and is therefore very much
    not something like a leaf package one can easily pretend doesn't exist.

    I think the idea of reporting them now was more about letting the bugs
    to be known "soon", more than starting to fix them "soon". Now we can
    forward the bugs upstream and some of the fixes will be present in the
    new upstream releases that we will start to upload after the release.

    I didn't challenge any reasons *for* filing. I'm giving a reason *not
    to*, in reply to someone who said wasn't seeing any.

    Also, as a random maintainer, despite the “Severity: minor”, seeing
    piles of FTBFS bug reports pop up all of a sudden in my maildir right
    before entering hard freeze does not bring any kind of warm, cozy
    feeling. Distraction and worry isn't quite something I enjoy. YMMV.


    Cheers,
    --
    Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
    D-I release manager -- Release team member -- Freelance Consultant

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEtg6/KYRFPHDXTPR4/5FK8MKzVSAFAmgn3hQACgkQ/5FK8MKz VSDaTBAAj1mZPFZkKYELNHXXZHzK8+8+PFnHSwsEAVFve5pXkicWr8R+fx+p24xN eJ5TvBsrjdKGWFWyzKUl+KczOYgpwleH+PI5OcPJeKr7FvNBN+TRgmutDginngM9 bMNghUJs/lcFFzWiUeZk7SjACmvvMq15vR6nNaArcMEHR1N3AnuTN5J8nbMoy+fj QJvi4KRG8qPhSdM6+wTUx0SubI6O9UqCk9jWExIbebMCMnP4uJoh6cd1ZTR/MZcD NNkXQbYJMqGRM3i7gmykA2hAiZ4EqbPfwqhrlInEwuXIdifnaVF+4bZJfOF8V08z rsYLdUFZf6560I09ceiDc093ovga3m2koLoGlbGDKEWDWZByIw3957OUO/W10wfE ie6+8+TnlraIPdhKx1KdKB39uJtfgdD7ok6UQVBNQNImvL/1RinaXX+mNOUUAw4q hfj2XsGdobKQgUkUfByMxYdABnqVCwcZU4ZlI2vZ4mVSRfSj/o9bZb6sZ92ddvPU qNRNwY74HFeCPCwq5pm8Ugm+/U2REos57G8oJpSM55s0S8ysB/o042ehhit+72IC AxSdwBOdrz31HgztK87CbZqybITj+64kMiPK/79jCYYXFPRpac68uxJ1bbRaHb9t 6/FCcMWTj4nyCaBRXnoSlO5Y9aC5HFqxFPFj5v+0SGohZRFw380=
    =gnnQ
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    *
  • From Santiago Vila@21:1/5 to All on Sat May 17 02:40:01 2025
    El 17/5/25 a las 1:13, Cyril Brulebois escribió:
    Soren Stoutner <soren@debian.org> (2025-05-05):
    Filing these bug reports sounds like a good idea to me. I don’t see
    any reason to wait as these will be severity:minor, so they won’t
    interfere with the trixie release.

    Filing now can trigger uploads to fix those minor bugs, meaning packages
    that absolutely do not meet requirements for freeze exceptions end up in unstable. Depending on circumstances, they might make migrations of other packages more complicated than they need to be.

    (I'll refrain from naming a particular example, but I'm aware of one of
    those without even searching for it, it just happened to show up on one of
    my specific “important for the release” radars…)

    Hi. Discussing about the right time to report those bugs does
    not make much sense anymore, because Lucas already reported them :-)

    They are here usertagged:

    https://bugs.debian.org/cgi-bin/pkgreport.cgi?users=lucas@debian.org;tag=ftbfs-shuffle

    Now we can only expect maintainers to act responsibly, and as you
    point out, be particularly careful about not affecting other packages.

    I think the idea of reporting them now was more about letting the bugs
    to be known "soon", more than starting to fix them "soon". Now we can
    forward the bugs upstream and some of the fixes will be present in
    the new upstream releases that we will start to upload after the
    release.

    Thanks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lucas Nussbaum@21:1/5 to Cyril Brulebois on Sat May 17 21:10:01 2025
    On 17/05/25 at 02:53 +0200, Cyril Brulebois wrote:
    Santiago Vila <sanvila@debian.org> (2025-05-17):
    El 17/5/25 a las 1:13, Cyril Brulebois escribió:
    Soren Stoutner <soren@debian.org> (2025-05-05):
    Filing these bug reports sounds like a good idea to me. I don’t see any reason to wait as these will be severity:minor, so they won’t interfere with the trixie release.

    Filing now can trigger uploads to fix those minor bugs, meaning packages that absolutely do not meet requirements for freeze exceptions end up in unstable. Depending on circumstances, they might make migrations of other packages more complicated than they need to be.

    (I'll refrain from naming a particular example, but I'm aware of one of those without even searching for it, it just happened to show up on one of
    my specific “important for the release” radars…)

    Hi. Discussing about the right time to report those bugs does not make
    much sense anymore, because Lucas already reported them :-)

    Given I mentioned spotting a resulting upload, it should be pretty clear
    that I do know that has happened.

    It seemed important to me to reply to the “I don't see any reason to wait” part (which I quoted, and kept above just in case), so that people might take that in consideration next time they want to MBF that late in
    the freeze.

    Now we can only expect maintainers to act responsibly, and as you
    point out, be particularly careful about not affecting other packages.

    The example I was alluding to builds a udeb, and is therefore very much
    not something like a leaf package one can easily pretend doesn't exist.

    I think the idea of reporting them now was more about letting the bugs
    to be known "soon", more than starting to fix them "soon". Now we can forward the bugs upstream and some of the fixes will be present in the
    new upstream releases that we will start to upload after the release.

    I didn't challenge any reasons *for* filing. I'm giving a reason *not
    to*, in reply to someone who said wasn't seeing any.

    Also, as a random maintainer, despite the “Severity: minor”, seeing
    piles of FTBFS bug reports pop up all of a sudden in my maildir right
    before entering hard freeze does not bring any kind of warm, cozy
    feeling. Distraction and worry isn't quite something I enjoy. YMMV.

    Sorry about that.

    I was kind-of expecting someone to answer "please wait" and I would have
    been fine with that. But I went ahead since
    1) nobody replied to say so;
    2) my own incentive for doing the MBF now was that I could file based on results I already had (not re-processing all packages to update
    results);
    3) severity:minor doesn't put much pressure in fixing the issues, so I
    expected maintainers of important packages to wait before uploading fixes,
    and thus the MBF to have no impact on release tasks.

    But your data point shows that (3) did not hold well... Sorry!

    Lucas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)