• Re: Moving apt (and hence bootstraps) from GnuPG to Sequioa (via gpgv-s

    From Julian Andres Klode@21:1/5 to Julian Andres Klode on Tue Dec 3 16:40:01 2024
    On Thu, Nov 21, 2024 at 09:16:20PM +0100, Julian Andres Klode wrote:
    I've just finished more or less, adjusting the APT test suite
    to test gpgv-sq. I plan to upload APT that tests gpgv-sq
    tomorrow. This ensures full compatibility between apt and
    gpgv-sq going forward.

    After that migrates to testing next week, I want to make
    the switch: APT by default should use gpgv-sq. Previous
    discussions with the security team did not reveal any
    blockers for that, despite the strenuous nature of
    security updates for Rust packages.

    This has been delayed. There's ongoing investigation into
    sqv and sqopv, which are smaller verifiers from Sequoia,
    measuring only 2MB and without an SQLite dependency, hence
    saving about 6MB.

    We also identified issues with keys that are using SHA-1
    binding signatures. I'm in the process of adding a crypto
    policy override to APT to re-allow these until 2026; but
    we also do not have a way to warn people about upcoming
    revocations, so lots of stuff will suddenly fail on a new
    year.

    I don't want unstable users to switch multiple times
    if possible.

    There's limitations

    1. gpgv-sq - this essentially mostly works as gpgv
    now, so we get the feedback we are used to

    2. sqopv - produces no usable feedback at this time
    if signature verification failed, and also does
    not respect SEQUOIA_CRYPTO_POLICY.

    3. sqv - produces very detailed feedback, but does not
    have supported for clearsigned files. This is _less_
    of a blocker: APT detaches signatures itself anyhow
    for historical reasons - gpgv lacked the ability to
    do so before, and we need to ensure that the verified
    bits actually match what we read later on.

    RHEL 10 is adopting sq and sqv as the tooling to ship;
    sqopv is technically much more powerful than sqv, though,
    this may be an odd decision that they went with sqv, given
    it also does not share an interface with `sq verify`.

    I want to spend a day on my 4 day weekend to see if I
    can whack sqopv and sqv into apt test suite compliance.


    My plan here is to use

    Depends: gpgv-from-sq | gpgv-sq | gpgv
    Recommends: gpgv-sq

    I'm considering adding architecture restrictions to
    allow opt-in on architectures, giving ports the ability
    to migrate when they deem ready.

    In light of the discussion above about sq(op)v, this
    may change into

    Depends: sqv [architecture list],
    gpgv [!architecture list]

    We had some further discussion about gpgv not being
    in widespread use and removing it from the default
    bootstrap possibly being a reasonable way forward.

    --
    debian developer - deb.li/jak | jak-linux.org - free software dev
    ubuntu core developer i speak de, en

    -----BEGIN PGP SIGNATURE-----

    wsG7BAABCgBvBYJnTyUXCRBvpFjdHbA/cUcUAAAAAAAeACBzYWx0QG5vdGF0aW9u cy5zZXF1b2lhLXBncC5vcmeLsWDZIEG+3W8Ai7iFhG+6o+hPVhFlf13lauI7QImb CBYhBE+1iKhMLd55p0x3h2+kWN0dsD9xAACeDA//cm/46zCf7oXPGdjwDlrAaxnQ Y9t0mscXwie3kR8k7HWqW4uIByHAzjRcD1Oc6P07K0dPsto3cVxNqbj23zFcIPFj KZ317ES9l2b2C0TX8PAsEqY4ZMTjFYV3HP8NIAdWeP5Tdjhwaz+wV47v5/qxVxpm uejo3po1PJ0nn9LFtyadsq9TqRLHGXEKkulGTWEsNdLJfPcgzXFZi2tHa9l0wIUM vBvKYU+/Nzqov0AfnxSA7t28zCGzkYHUYM74wszgtPtWgW/eZKIUHS1qBZC8n/qj vX90njOLovhkedurZ0aRme0DLHQHJt278P5sfBSxIncm2SWG0/HsNNZEXsl94rHO VMhagFFHhcfl6Y6V8+FxvVoMfRSGutQu/eX2z9LyoUXATNQvql10+ZugGmfC2/yl 4ZH9ql+jGYfYTXWGv1BdqV69QWyoM9TOGsBvGWM6be6ONwTzFfB6h4hoyVidizBF zBZbmSPwwFdMEtF05KMvMwFflK69iEaI0kzZ3urtDkyQ2DqCWWveOVtE5vxetrq8 79A5yq7qFAGT1JIQ/eOAVvizqMflWTZSPTSqzlgbPGSnZ/9SYU0L4PK4jX7sOsBR duKborr/P1TfteaJBb2on+lnPsXJrjpP2CdaAznxms3aWChrw9xmWzSTTH
  • From Julian Andres Klode@21:1/5 to Julian Andres Klode on Wed Dec 18 00:10:01 2024
    On Tue, Dec 03, 2024 at 04:34:52PM +0100, Julian Andres Klode wrote:
    On Thu, Nov 21, 2024 at 09:16:20PM +0100, Julian Andres Klode wrote:
    I've just finished more or less, adjusting the APT test suite
    to test gpgv-sq. I plan to upload APT that tests gpgv-sq
    tomorrow. This ensures full compatibility between apt and
    gpgv-sq going forward.

    After that migrates to testing next week, I want to make
    the switch: APT by default should use gpgv-sq. Previous
    discussions with the security team did not reveal any
    blockers for that, despite the strenuous nature of
    security updates for Rust packages.

    This has been delayed. There's ongoing investigation into
    sqv and sqopv, which are smaller verifiers from Sequoia,
    measuring only 2MB and without an SQLite dependency, hence
    saving about 6MB.

    An sqv backend is now available in apt-team/apt!409 and in
    experimental in apt 2.9.17+exp1.

    Note that the experimental upload only supports architectures
    with sqv available. There is no fallback yet.

    The plan is to detect if sqv is available at build time, by
    build-depending on sqv for the correct set of architectures,
    and then generate a `Depends: sqv` for those architectures,
    and `Depends: gpgv` for other (ports) architectures.

    The sqv binary is about 2MB large when optimized for size,
    and provides good feedback when a key cannot be verified.
    --
    debian developer - deb.li/jak | jak-linux.org - free software dev
    ubuntu core developer i speak de, en

    -----BEGIN PGP SIGNATURE-----

    wsG7BAABCgBvBYJnYgL4CRBvpFjdHbA/cUcUAAAAAAAeACBzYWx0QG5vdGF0aW9u cy5zZXF1b2lhLXBncC5vcmey32GfIcp5vyNv79BPinH33a2IMMyAYlEgrjOf5k6X bxYhBE+1iKhMLd55p0x3h2+kWN0dsD9xAAAJ+BAAlvusxPhOk8coNxvdryW7Btii yucqnzAJCtfzQvSV8V/uqun/JgviSqKbsn++A4X3BVNAtf5u2HYXoAsbFRp4nmCX GQaC2mvX/VXOKJSMAHyO/GgHBpk3moxW4X2lMfElEZ6BTGo4wF/HUiE5Vn0hma1a ooT5cej6qIV2oOUxgvAgfUWKd7QqsLhhrFAS9wG5uZWg58zpzdwTjZApc/6hgC53 lfQeaEStBkpEW7hEWNacixTUcOVSmdxLylk1m++0wjQgtYWfwni8/shoo4nUAIXL KsbrVA6MoXPpp2zuNZah6bdL3ZZK33C2mRKCzAMIg1OLvgNK/3rWXPk5warnl9QR dUvOkgCyvhyUQyagLZNXyOUfA43lxxnao8PsOgQF2CWzqmoFFDpAOKI0JCRiiDb/ xuy0tKvejt8PJh3IF4iosVWX/OVzrPr8lN0ukCOjbAvV1SDu+XnbfegnuMGW45KB sL8uAauplT3qORRhMxXWKyc6wJBhRMvEyno/Ye8fc9EpHZqny9pgXKOfYTYFOvDS p9g+EQUHRqe9vgRuS6z2IizuZzAs1Ln1Ti+xYb/JCr4yQ+0/jp4wK8S32LkTRfBi Tp7oN3jA5VNnp1F3HJ5ST5UtQVkyHC2jl3bXJo1whM01dN7SgQzTyj9FIu
  • From Julian Andres Klode@21:1/5 to Julian Andres Klode on Mon Dec 23 12:50:01 2024
    On Mon, Dec 23, 2024 at 12:29:09PM +0100, Julian Andres Klode wrote:
    On Wed, Dec 18, 2024 at 12:02:18AM +0100, Julian Andres Klode wrote:
    On Tue, Dec 03, 2024 at 04:34:52PM +0100, Julian Andres Klode wrote:
    On Thu, Nov 21, 2024 at 09:16:20PM +0100, Julian Andres Klode wrote:
    I've just finished more or less, adjusting the APT test suite
    to test gpgv-sq. I plan to upload APT that tests gpgv-sq
    tomorrow. This ensures full compatibility between apt and
    gpgv-sq going forward.

    After that migrates to testing next week, I want to make
    the switch: APT by default should use gpgv-sq. Previous
    discussions with the security team did not reveal any
    blockers for that, despite the strenuous nature of
    security updates for Rust packages.

    This has been delayed. There's ongoing investigation into
    sqv and sqopv, which are smaller verifiers from Sequoia,
    measuring only 2MB and without an SQLite dependency, hence
    saving about 6MB.

    An sqv backend is now available in apt-team/apt!409 and in
    experimental in apt 2.9.17+exp1.

    Note that the experimental upload only supports architectures
    with sqv available. There is no fallback yet.

    The plan is to detect if sqv is available at build time, by
    build-depending on sqv for the correct set of architectures,
    and then generate a `Depends: sqv` for those architectures,
    and `Depends: gpgv` for other (ports) architectures.

    The sqv binary is about 2MB large when optimized for size,
    and provides good feedback when a key cannot be verified.

    The Sequoia sqv backend is now the default backend in unstable
    for architectures that have it (all release architectures, most
    ports).

    2.9.19 also replaces internal GnuTLS and gcrypt use with OpenSSL,
    and all use of GnuPG in the test suite with Sequoia's `sq` command.

    There is a backwards-incompatible change: Signed-By can no
    longer contain an exact subkey match (suffix "!"). That
    information is - rightly so - not available in the sqv
    output.

    Space consumption, with apt from experimental:

    105M experimental.min.tar
    192M experimental.tar
    114M unstable.min.tar
    196M unstable.tar

    i.e. we see a 9MB saving for essential+apt, and a 4MB saving
    for a default mmdebstrap. Something still pulls in gpgv there
    which is unfortunate, we lack a 5MB savings.

    More savings can be achieved by building sqv using openssl,
    then we stop pulling in nettle.


    --
    debian developer - deb.li/jak | jak-linux.org - free software dev
    ubuntu core developer i speak de, en

    -----BEGIN PGP SIGNATURE-----

    wsG7BAABCgBvBYJnaU4gCRBvpFjdHbA/cUcUAAAAAAAeACBzYWx0QG5vdGF0aW9u cy5zZXF1b2lhLXBncC5vcmf1uy4JvnHG278Lo/rk+lGlziT92bRvtcsIWUlMZTxg VRYhBE+1iKhMLd55p0x3h2+kWN0dsD9xAADlOA//RWpl2VVsG5Fcf1GXcj9FAcu5 WkzMJSeby8QNaLrUwRE7vL6nF6bR4BrPF0gz29tIlVxxV0pvS+5L6e2VJ23qgZrj 7xY1tvEowYal2MizomiptPCgR8dmo7bsGRDPAfhG39KmB3Z5NqGHEyRPZgdtJKAh yciQY8evosGQBU/DwxOGHg3xzIwX6FIy/cxiw+VqspbX2PpAzh6jo4noi0qBsupo H4u1RiIvv+sTM9+N/0QX2+84LcdGDhLcOjdsizBBv5cKTG9JTQ7cSnG6DnFUQtKr 7UWe/Zxxo0UC37vx9Vxxnw9/RyuIG0Zi2qFeaXqJlMJgCKnGzE2Hl6kBbKhIw7rN Dr+myzPw5Gq7vWKd9tQIV6Z3UVOnlKTQ/UrVKflzdBKq9qZE65o3CbTS8+XwaCWL 2HFeTYWWfITZR/Kl1dPfgz/7OIy65Hi+hsT8zVhDRCfwta+eBdz8kNrvHNxM3kXE OeLI4r6f0tvvWaMTNAKGPJv4nObO8eZ4S4vzDzQ6onUWvmB+gfQDxbJ7U8ijrvhe kExN+r0JZMqSh9keYOdLPMOl4uUK1m1nllZaeDdqhqBR1FSLjDgFs6gxEEq7ndqW Dz14T5ZEC6uSYXQwlxHxolncyggos4IasWvh/w7kwJBmuOw3UKYATHj09J
  • From Julian Andres Klode@21:1/5 to Julian Andres Klode on Mon Dec 23 12:30:01 2024
    On Wed, Dec 18, 2024 at 12:02:18AM +0100, Julian Andres Klode wrote:
    On Tue, Dec 03, 2024 at 04:34:52PM +0100, Julian Andres Klode wrote:
    On Thu, Nov 21, 2024 at 09:16:20PM +0100, Julian Andres Klode wrote:
    I've just finished more or less, adjusting the APT test suite
    to test gpgv-sq. I plan to upload APT that tests gpgv-sq
    tomorrow. This ensures full compatibility between apt and
    gpgv-sq going forward.

    After that migrates to testing next week, I want to make
    the switch: APT by default should use gpgv-sq. Previous
    discussions with the security team did not reveal any
    blockers for that, despite the strenuous nature of
    security updates for Rust packages.

    This has been delayed. There's ongoing investigation into
    sqv and sqopv, which are smaller verifiers from Sequoia,
    measuring only 2MB and without an SQLite dependency, hence
    saving about 6MB.

    An sqv backend is now available in apt-team/apt!409 and in
    experimental in apt 2.9.17+exp1.

    Note that the experimental upload only supports architectures
    with sqv available. There is no fallback yet.

    The plan is to detect if sqv is available at build time, by
    build-depending on sqv for the correct set of architectures,
    and then generate a `Depends: sqv` for those architectures,
    and `Depends: gpgv` for other (ports) architectures.

    The sqv binary is about 2MB large when optimized for size,
    and provides good feedback when a key cannot be verified.

    The Sequoia sqv backend is now the default backend in unstable
    for architectures that have it (all release architectures, most
    ports).

    2.9.19 also replaces internal GnuTLS and gcrypt use with OpenSSL,
    and all use of GnuPG in the test suite with Sequoia's `sq` command.

    There is a backwards-incompatible change: Signed-By can no
    longer contain an exact subkey match (suffix "!"). That
    information is - rightly so - not available in the sqv
    output.
    --
    debian developer - deb.li/jak | jak-linux.org - free software dev
    ubuntu core developer i speak de, en

    -----BEGIN PGP SIGNATURE-----

    wsG7BAABCgBvBYJnaUmDCRBvpFjdHbA/cUcUAAAAAAAeACBzYWx0QG5vdGF0aW9u cy5zZXF1b2lhLXBncC5vcme9vM9CGwsbqLIkTUXRuKcI2jHLd7LfdvmIyMwq3fM4 VhYhBE+1iKhMLd55p0x3h2+kWN0dsD9xAABPYw//UzwIl9Xil1Hj49Q3di6FAnlE X71jSHfOrmp1AjAr5fKhbJ9o/doUQPkcpFsRmEPaFnmxPObMo3wq0yadK1jXgopg 6wihV/PYGV4dYODqbImGBih5WZB1nUNK9WV5iX3+19q7fsiiN59TAE3SWAncRX5l uml930XKnzXGZS8/RXX2AB2Cli8HV9yD5Nuvx8TO2ROmaSEL5r+ugzgJ3FSsdLWA pePPBywMQSjTGMoATRRb9W6O3Cxa69j52GSRTB4lJ9BtcSZHUcS6YU8mis7Vcyvv O30D0K87sYFerRcOQ8T4Wrjh2Ho3eG2YtO2uCDP2cGKahouXHbv3nP3tm5usY7/+ /XlZL/JYAPCMXjJO70yOYDKy0TSzEmrOc6zzxRsnDfDYXT3fZSqRgXtRPx4JhDOu 9fwDgmUOrDacLJsAvikc7Wx/A+26npbZTL2hHJcLsHXgIffVvpR4cmMwlPM21i5d PaFNmcvye4kFuSCdWEIGmQc8lWKIQ5Im1RADjBl8UR378KrU23g6kODPbVO+KOAK VGiRIgD1KT92B1Z3g6P2UtqcGzy//9v/b8pRt8/28hUjaS5A+gw5mF3Tk3jxrXNR kTzEV7JHe+rrqgGicRgZbLzEvZlJQpH3Pu8ybgAtcl+OOhj8+kpN7Szbh6
  • From Holger Levsen@21:1/5 to Julian Andres Klode on Mon Dec 23 13:20:01 2024
    On Mon, Dec 23, 2024 at 12:48:50PM +0100, Julian Andres Klode wrote:
    i.e. we see a 9MB saving for essential+apt, and a 4MB saving
    for a default mmdebstrap.

    very nice!

    Something still pulls in gpgv there
    which is unfortunate, we lack a 5MB savings.

    More savings can be achieved by building sqv using openssl,
    then we stop pulling in nettle.

    /me nods, something for 2025.

    which reminds me something to share here: in the last weeks I've enabled
    size optimisations for chameleon and sqv bringing down the sizes (in
    bytes) from

    6208216 /usr/bin/gpgv-sq
    10780992 /usr/bin/gpg-sq
    2958984 /usr/bin/sqv

    to

    4533016 /usr/bin/gpgv-sq
    7319936 /usr/bin/gpg-sq
    1718816 /usr/bin/sqv

    with hardly an impact on performance.

    Doing the same for sq is my agenda as well as packaging sq 1.0 :)


    --
    cheers,
    Holger

    ⢀⣴⠾⠻⢶⣦⠀
    ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
    ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
    ⠈⠳⣄

    It's the end of the world as we know it - and I feel fine.

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEuL9UE3sJ01zwJv6dCRq4VgaaqhwFAmdpVO8ACgkQCRq4Vgaa qhzsdxAAsqFmegeE4tWlNcfTuNAJ7LJCFKdQPjurRxyds4toW538preV/EaZoFQQ k7UFDkHf2sbuW9uyVSh51GuEagibTqSIl/UDPY8hXllvpw+ilVvG0fXExYi7pXQa JIAahiWpPmRgJJM8drfCrwZiZmUT3QpfnRbhKrjr1i9JfugFwNLVi45E0N15L9n4 BRVH90jc6PGFNBc40+w+YJq5H+A4cmBPw1zCw+/yEEq1UuERA3mTo9R3XlborVm2 iTIKmx6CYalEZM0SBpIMTL/ejqfMkeQlrHiTzpFZTs40jEsjndBo1Vvqnj4ujD0D mKIk0/MK9edvwpc+qD85nfkzOFpQUpRECE2t09l2lqB/bOhQEkfntfx5MCJJ0Gjk YH1PJRiTGcvdtrUdrpTR6YQ0PK6fk40b12JbyuxV1kuzHrk9bvGFhSb8ZmcpQDU2 RPKeZ7bzNEl5BiVzvzgUroojwFY7tSG/yRHLJwllKmWgeQarZ/SCymsqNDVJyerr pZmCmaevK3n8WfKSkfkG7SGK11zokxb/d5vUlcC3ice6EeUhrNce8OuiLPrNoJl1 mPWg04Hbon+aggLMg78h0rbYT6GFEQwgPT80VWDn5ensDeKRNAlbwe
  • From Chris Hofstaedtler@21:1/5 to All on Mon Dec 23 13:30:02 2024
    * Julian Andres Klode <jak@debian.org> [241223 12:49]:
    Something still pulls in gpgv there
    which is unfortunate, we lack a 5MB savings.

    dpkg-dev Depends: gpgv | sq | ...

    That seems odd. Maybe it wants gpgv | sqv | ...
    instead?

    If its not dpkg-dev, I don't see whats pulling gpgv.

    Chris

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Guillem Jover@21:1/5 to Chris Hofstaedtler on Mon Dec 23 14:00:01 2024
    Hi!

    On Mon, 2024-12-23 at 13:20:39 +0100, Chris Hofstaedtler wrote:
    * Julian Andres Klode <jak@debian.org> [241223 12:49]:
    Something still pulls in gpgv there
    which is unfortunate, we lack a 5MB savings.

    I think that would be gpgv being Priority: important, which makes
    debootstrap and friends pull it in by default. I guess that might
    need to be swapped now.

    dpkg-dev Depends: gpgv | sq | ...

    That seems odd. Maybe it wants gpgv | sqv | ...
    instead?

    I do have a branch to add support for sqv, should get in with the next
    dpkg upload. And probably can now swap the order of preference there
    too.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Julian Andres Klode@21:1/5 to Guillem Jover on Mon Dec 23 14:10:01 2024
    On Mon, Dec 23, 2024 at 01:57:42PM +0100, Guillem Jover wrote:
    Hi!

    On Mon, 2024-12-23 at 13:20:39 +0100, Chris Hofstaedtler wrote:
    * Julian Andres Klode <jak@debian.org> [241223 12:49]:
    Something still pulls in gpgv there
    which is unfortunate, we lack a 5MB savings.

    I think that would be gpgv being Priority: important, which makes
    debootstrap and friends pull it in by default. I guess that might
    need to be swapped now.

    This is being tracked in 1091200 (the demotion). I didn't suggest
    a swap, after all, apt's dependencies will pull in the right one.



    dpkg-dev Depends: gpgv | sq | ...

    That seems odd. Maybe it wants gpgv | sqv | ...
    instead?

    I do have a branch to add support for sqv, should get in with the next
    dpkg upload. And probably can now swap the order of preference there
    too.

    Sweet!

    --
    debian developer - deb.li/jak | jak-linux.org - free software dev
    ubuntu core developer i speak de, en

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)