• [gentoo-dev] [PATCH 0/3] cargo.eclass: Trivial crate replacements

    From kangie@gentoo.org@21:1/5 to All on Mon Nov 25 04:40:01 2024
    From: Matt Jolly <kangie@gentoo.org>

    Updating vulnerable or outdated crates in Rust ebuilds is not easy.
    This patchset aims to reduce the developer effort and knowledge
    required to substitute crates into a Rust build process.

    We do this by:

    - Enabling `paths = ['/foo/bar/baz']` style global replacements.
    This useful for ebuilds that vendor their crates but has some
    substantial limitations. This is required to (e.g.) build an
    older dev-lang/rust against a modern system with OpenSSL 3.

    - Providing a helper function that takes advantage of the eclass
    replacing crates.io with an offline repository to run
    `cargo update --offline` and update the Lockfile and crate
    metadata to include provided crate updates.

    There is some room for additional work to arbitrarily patch dependency
    crates (including checksum updates) and ensure that they are suitable
    for dependency resolution, however where a vulnerable (or otherwise
    broken) crate needs to be replaced with an updated version from
    crates.io this should prove suitable for most use cases.

    The following GitHub Pull Request has been opened for review
    feedback: https://github.com/gentoo/gentoo/pull/39464

    Matt Jolly (3):
    cargo.eclass: add trivial crate overrides
    dev-lang/rust{,-bin}: Add 1.54.0
    app-antivirus/clamav: example of trivial crate replacement

    app-antivirus/clamav/Manifest | 2 +-
    ...1.0.7-r1.ebuild => clamav-1.0.7-r2.ebuild} | 8 +-
    dev-lang/rust-bin/Manifest | 34 ++
    dev-lang/rust-bin/rust-bin-1.54.0.ebuild | 188 ++++++
    dev-lang/rust/Manifest | 4 +
    ...nore-broken-and-non-applicable-tests.patch | 75 +++
    .../1.49.0-gentoo-musl-target-specs.patch | 164 +++++
    .../rust/files/1.53.0-rustversion-1.0.5.patch | 234 ++++++++
    .../rust/files/1.54.0-parallel-miri.patch | 43 ++
    .../files/llvm/12/cstdint-signals-h.patch | 24 +
    dev-lang/rust/rust-1.54.0.ebuild | 560 ++++++++++++++++++
    eclass/cargo.eclass | 115 +++-
    eclass/rust.eclass | 2 +
    13 files changed, 1445 insertions(+), 8 deletions(-)
    rename app-antivirus/clamav/{clamav-1.0.7-r1.ebuild => clamav-1.0.7-r2.ebuild} (99%)
    create mode 100644 dev-lang/rust-bin/rust-bin-1.54.0.ebuild
    create mode 100644 dev-lang/rust/files/1.47.0-ignore-broken-and-non-applicable-tests.patch
    create mode 100644 dev-lang/rust/files/1.49.0-gentoo-musl-target-specs.patch
    create mode 100644 dev-lang/rust/files/1.53.0-rustversion-1.0.5.patch
    create mode 100644 dev-lang/rust/files/1.54.0-parallel-miri.patch
    create mode 100644 dev-lang/rust/files/llvm/12/cstdint-signals-h.patch
    create mode 100644 dev-lang/rust/rust-1.54.0.ebuild

    --
    2.47.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From kangie@gentoo.org@21:1/5 to All on Mon Nov 25 04:40:01 2024
    From: Matt Jolly <kangie@gentoo.org>

    Updating vulnerable (or otherwise outdated) crates in Rust ebuilds
    is painful. Generally speaking, there are 5 options:

    - Run `cargo update` to fetch new versions from the web.
    This is obviously not suitable for use in Portage.
    - Patch the software via Portage to accept a non-vulnerable crate.
    This is a reasonable option when the package is not too complex
    but still requires significant developer effort and some familiarity
    with Cargo. In the case of complex patches this may not be feasible,
    or require the generation of a dependency tarball.
    - [patch] the source (repository) in Cargo.toml. This enables the
    targeting of specific crates, but does not allow the replacement
    of only a specific version in the depgraph.
    - [replace] a particular crate:version in the Cargo.toml. This
    enables the targeting of a particular version with an arbitrary
    path however the replacement crate must *have the same version*
    as the one being overridden.
    - `paths = [...]` overrides: pass an array of paths to directories that
    contain a Cargo.toml. Cargo will override any crate with the same package name
    arbitrarily, ignoring the lock file and versions; typically used for testing.
    Is applied via ${CARGO_HOME}/config.toml (i.e. globally)

    This commit:

    - Implements the `paths` overrides, which will work even when
    Cargo is configured to use a vendored directory. This is not a 'smart'
    replacement and care must be taken to ensure that all versions of
    the crate in use are compatible (`cargo tree` will help).

    - Provides a helper which runs `cargo --update --offline` against
    ${ECARGO_VENDOR} (where ${CRATES} are unpacked). This enables the
    replacement of vulnerable versions in ${CRATES}. It is up to the
    consumer to ensure that only the desired crates are being replaced
    and that package behaviour does not change.

    - Adds a new `CARGO_BOOTSTRAP` variable which enables packages to
    ignore the minimum version requirement of the eclass. This is only
    used for bootstrapping Rust; if it's being used in any non
    dev-lang/rust ebuilds be sure that you have a good reason.

    Resources:
    - https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html
    - https://github.com/rust-lang/cargo/issues/3308

    Signed-off-by: Matt Jolly <kangie@gentoo.org>
    ---
    eclass/cargo.eclass | 115 +++++++++++++++++++++++++++++++++++++++++---
    1 file changed, 109 insertions(+), 6 deletions(-)

    diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
    index 95ff317e1f21..a49ef818a351 100644
    --- a/eclass/cargo.eclass
    +++ b/eclass/cargo.eclass
    @@ -7,6 +7,7 @@
    # @AUTHOR:
    # Doug Goldstein <cardoe@gentoo.org>
    # Georgy Yakovlev <gyakovlev@gentoo.org>
    +# Matt Jolly <kangie@gentoo.org>
    # @SUPPORTED_EAPIS: 8
    # @PROVIDES: rust
    # @BLURB: common functions and variables for cargo builds
    @@ -37,8 +38,10 @@ case ${EAPI} in
    if [[ -n ${RUST_MIN_VER} ]]; then
    # This is _very_ unlikely given that we leverage the rust eclass but just in case cargo requires a newer version
    # than the oldest in-tree in future.
    - if ver_test "${RUST_MIN_VER}" -lt "${_CARGO_ECLASS_RUST_MIN_VER}"; then
    - die "RUST_MIN_VERSION must be at least ${_CARGO_ECLASS_RUST_MIN_VER}"
    + if [[ -z ${CARGO_BOOTSTRAP} ]]; then
    + if ver_test "${RUST_MIN_VER}" -lt "${_CARGO_ECLASS_RUST_
  • From kangie@gentoo.org@21:1/5 to All on Mon Nov 25 04:40:01 2024
    From: Matt Jolly <kangie@gentoo.org>

    These are the the old ebuilds, updated to be slotted.

    Other changes:

    - `USE=rls` was dropped at some point, this is now
    hard-enabled.
    - As the LLVM is far too old for Gentoo we are using the bundled LLVM
    exclusively.

    In particular, dev-lang/rust:1.54.0 forms an important part of the
    upcoming 'bootstrap Rust without Rust binaries' path.

    Bug: https://bugs.gentoo.org/943706
    Signed-off-by: Matt Jolly <kangie@gentoo.org>
    ---
    dev-lang/rust-bin/Manifest | 34 ++
    dev-lang/rust-bin/rust-bin-1.54.0.ebuild | 188 ++++++
    dev-lang/rust/Manifest | 4 +
    ...nore-broken-and-non-applicable-tests.patch | 75 +++
    .../1.49.0-gentoo-musl-target-specs.patch | 164 +++++
    .../rust/files/1.53.0-rustversion-1.0.5.patch | 234 ++++++++
    .../rust/files/1.54.0-parallel-miri.patch | 43 ++
    .../files/llvm/12/cstdint-signals-h.patch | 24 +
    dev-lang/rust/rust-1.54.0.ebuild | 560 ++++++++++++++++++
    eclass/cargo.eclass | 2 +-
    eclass/rust.eclass | 2 +
    11 files changed, 1329 insertions(+), 1 deletion(-)
    create mode 100644 dev-lang/rust-bin/rust-bin-1.54.0.ebuild
    create mode 100644 dev-lang/rust/files/1.47.0-ignore-broken-and-non-applicable-tests.patch
    create mode 100644 dev-lang/rust/files/1.49.0-gentoo-musl-target-specs.patch
    create mode 100644 dev-lang/rust/files/1.53.0-rustversion-1.0.5.patch
    crea
  • From James Le Cuirot@21:1/5 to kangie@gentoo.org on Mon Nov 25 21:20:01 2024
    On Mon, 2024-11-25 at 13:35 +1000, kangie@gentoo.org wrote:
    From: Matt Jolly <kangie@gentoo.org>

    These are the the old ebuilds, updated to be slotted.

    Other changes:

    - `USE=rls` was dropped at some point, this is now
      hard-enabled.
    - As the LLVM is far too old for Gentoo we are using the bundled LLVM
      exclusively.

    In particular, dev-lang/rust:1.54.0 forms an important part of the
    upcoming 'bootstrap Rust without Rust binaries' path.

    Bug: https://bugs.gentoo.org/943706
    Signed-off-by: Matt Jolly <kangie@gentoo.org>

    You've lost some lines at the start of dev-lang/rust's src_configure, the filter-lto line and the cross-compiler lines I added the other day. Was that intentional?

    -----BEGIN PGP SIGNATURE-----

    iQJFBAABCAAvFiEEPxcZ3tkwcedKm2a8EiZBXQDdMTcFAmdE2lcRHGNoZXdpQGdl bnRvby5vcmcACgkQEiZBXQDdMTe42w//TO2EIzpRudXoNYSiyMAF19Cu/EsHna4G h4nzYNVzrwSaZK22x0TaXnUHF8Iedu7pJgE/Sfsg1bfEJae5Ccmr9VOjXtaT2yOd AothVR9nfCUv5xp5CZ4OL+pLOGt/mVQXx1GlcOWVunHEmdmaDyZ2lZp3ghSg+hv9 oyKwyalsnJqPFg4bc16cwYDaveRKaIo68MA45YV375BGu+/zNmpqSdrgEqkyBzO5 UQbe8Vpo+lzgDah96jODtbzF3jK5LiWadHKiwO7o79Pc4vk4dtwu1BtZXTYZ+sV6 qDOnIYLWiwTnpbdHhnqFiw9+qPsADIcXtuXtESSZpI9qPazFHuJV+PJlvhgN5yKD Al50A6Q7sRKq6Kwtus6Kn22Ek4WoJiS3b/A9qRiZyYCkn2wD2P0Ce3mk4ZmL+o5y UiVDcVgshtjO8loZAyXItRY5XpptMjwdEPesQeNFYDJnfC7MojjwCbKS93zd+EJq U6+5Un4pVz3iwGgd6DKoqlHT18MceobnVtCuTQilgdnnv0orVlmIEYftzFuFJ6hf CVVRF61pyvkqflG3Oszezk+VlcU8VHj+LfVuD1YDaAXOM8RKupEVzjXXB1F9b8ut C+MIoA/KzYSP5j9sT0KtMSAHqKgNuwU0ztj6RNIE30dxl+GkXDDRJiCKOEq2WtiH
    A4OT6E0Y0YU=
    =om7l
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Matt Jolly@21:1/5 to James Le Cuirot on Mon Nov 25 22:40:01 2024
    Hi,

    On 26/11/24 06:13, James Le Cuirot wrote:

    You've lost some lines at the start of dev-lang/rust's src_configure, the filter-lto line and the cross-compiler lines I added the other day. Was that intentional?

    This ebuild was modified from 1.54.0 as it was when it was removed from
    the tree, and this branch didn't include those updates until a rebase
    before pushing.

    This is mostly intended to be a the beginnings of a 'binary-Rust-less' bootstrap path using `mrustc`, I can't really see any situation where
    the ability to cross-compile a very old rust adds value.

    I'll take some time today to implement, regardless.

    Cheers,

    Matt

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From James Le Cuirot@21:1/5 to Matt Jolly on Mon Nov 25 23:40:01 2024
    On Tue, 2024-11-26 at 07:36 +1000, Matt Jolly wrote:
    Hi,

    On 26/11/24 06:13, James Le Cuirot wrote:

    You've lost some lines at the start of dev-lang/rust's src_configure, the filter-lto line and the cross-compiler lines I added the other day. Was that
    intentional?

    This ebuild was modified from 1.54.0 as it was when it was removed from
    the tree, and this branch didn't include those updates until a rebase
    before pushing.

    This is mostly intended to be a the beginnings of a 'binary-Rust-less' bootstrap path using `mrustc`, I can't really see any situation where
    the ability to cross-compile a very old rust adds value.

    I'll take some time today to implement, regardless.

    Ah sorry, ignore me then! I wasn't paying attention and thought this was a new version. There's a fair chance it won't cross-compile anyway, especially if an older LLVM is involved.

    -----BEGIN PGP SIGNATURE-----

    iQJFBAABCAAvFiEEPxcZ3tkwcedKm2a8EiZBXQDdMTcFAmdE+q8RHGNoZXdpQGdl bnRvby5vcmcACgkQEiZBXQDdMTcivA//VpF5RmTz5dFscJD20CQ3qqmyzNHPcQHM +u/xxxOLtUq4dHqKOmhMrFmUi9G+S7wWjBifYx+EfF0Ga2vY3pNCvkDcNJnjROlp p49m2vFjcmGACXYuhb+3TfsPW/WmjXN8kqcauRbSbDNX1poGkx/yORwXurpRJKmG glewyZWyErTH2hRfkbZalv3tQGLFvA6Hw7xV3heJJrq33VBNrCllokFo0w/3ElNt 9Bolscx/peWedtHQEpI+1L8CT/R4YrEYarFpMsMGXpoNsJ/cV3VVHb9i2PHMFOTI L80uPf73Cnkp6hmByZ+eFjm18PMWkViCojr7ZOtRIt68DEoMIn2oPBXwNyFobiFL yPwqbEPPhWiSwIPw4Cin6sSxqsaN1dIATgrDalAIjYnwem1Z39yfRu3nvQ3iOMnM Wb357kTknwUezZWhOWm3S5g8/w99hnri03XySPRoqTuIIXDSjaN0ju7/O/IBx8p0 rh2+ZadJ9W+c+eunUaKGGbVUBE9kTO9sCoiZHkz8QIoJtTiEPWeV78EH0U4+alaw qXdZz2gTU4JQCeY/0LXLn7jqZ4PPKWI0N87Vgi6qWmvA6U5MazUOS5UuAcIFmzo5 Zrs+29ZRKl5NN1h7B3paEI7gdj37SCyWPsqRilsbiqn07+4MhkHoLNRU+UMtjLjc
    ac4cOAY9YVY=
    =jmVR
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)