• Bug#1088104: partman-auto-raid: Recipe delimiter parsing shortcoming re

    From Gaudenz Steinlin@21:1/5 to All on Wed May 14 17:00:01 2025
    XPost: linux.debian.maint.boot

    Hi

    I would really like this patch to be included into
    partman-auto-raid. The same bug also prevents device paths with a
    dot. This is common in /dev/disk/by-path symlinks. It would be
    great to be able to use such symlinks to avoid issues with
    different device enumerations depending on how man slots are
    actually populated with disks.

    We currently have a bunch of systems with 2 NVMe devices used in a
    RAID configuration as system disks. Some of these systems contain
    2 additional devices. The system disks are always plugged into the
    same physical ports. The 2 additional devices cause the device
    numbering of the system disks to change.

    With 2 disk:
    $ ls -l /dev/disk/by-path/ total 0 lrwxrwxrwx 1 root root 13 Mai
    14 15:52 pci-0000:61:00.0-nvme-1 -> ../../nvme0n1 lrwxrwxrwx 1
    root root 13 Mai 14 15:52 pci-0000:a4:00.0-nvme-1 -> ../../nvme1n1

    With 4 disks:
    $ ls -l /dev/disk/by-path/ total 0 lrwxrwxrwx 1 root root 13 Mai
    14 15:52 pci-0000:61:00.0-nvme-1 -> ../../nvme0n1 lrwxrwxrwx 1
    root root 13 Mai 14 15:52 pci-0000:a1:00.0-nvme-1 -> ../../nvme1n1
    lrwxrwxrwx 1 root root 13 Mai 14 15:52 pci-0000:a2:00.0-nvme-1 -> ../../nvme2n1 lrwxrwxrwx 1 root root 13 Mai 14 15:52
    pci-0000:a4:00.0-nvme-1 -> ../../nvme3n1

    It would be nice to configure the RAID with stable device namesl
    like this:

    /dev/disk/by-path/pci-0000:61:00.0-nvme-1#/dev/disk/by-path/pci-0000:a4:00.0-nvme-1

    but this currently breaks because of the dot in the device name.

    Gaudenz

    --
    PGP: 836E 4F81 EFBB ADA7 0852 79BF A97A 7702 BAF9 1EF5

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iJIEARYKADoWIQS0SHD2ODl+Y8zt8TtkDjbn9vx/EgUCaCSsnxwcZ2F1ZGVuekBk dXJjaGVpbmFuZGVydGFsLmNoAAoJEGQONuf2/H8SZx4A/ik9bpECezxxLdNPSk/b AnPzbudRD+5245Pfu/nEYaaHAQDgiknjXMnZdDZBn6K/4rxPEgcyg1h4M4OkRvs2
    uB7bBQ==pzbu
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Cyril Brulebois@21:1/5 to All on Wed May 14 22:00:01 2025
    XPost: linux.debian.maint.boot

    Hi,

    Adding back the submitter.

    Gaudenz Steinlin <gaudenz@durcheinandertal.ch> (2025-05-14):
    I would really like this patch to be included into partman-auto-raid.
    The same bug also prevents device paths with a dot. This is common in /dev/disk/by-path symlinks. It would be great to be able to use such
    symlinks to avoid issues with different device enumerations depending on
    how man slots are actually populated with disks.

    […]

    Feel free to add this to the wishlist, without any promises though.

    https://wiki.debian.org/DebianInstaller/Trixie/Wishlist

    I haven't tried to fully understand the patches and its implications, but
    that might be doable even if we're getting late in the release cycle.
    Right now, a release is getting prepared and adding moving pieces is a big no-no.

    Johannes, we don't have anyone working on d-i fulltime (or anything
    remotely close to it), that's usually the main reason why merge requests
    or patches attached to bug reports might linger. Tut mir leid.


    Cheers,
    --
    Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
    D-I release manager -- Release team member -- Freelance Consultant

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEtg6/KYRFPHDXTPR4/5FK8MKzVSAFAmgk82EACgkQ/5FK8MKz VSD1IBAAhNkM0mqozIRzdyUnNP0HR9rMWkOLHVpvKJviSxhuG0j/EbETvXRPw3eh DJRTE2s9UaKFxAH1S5gFJj3jd8UOCkd3+2xFqoTZJ+8PBDaQg1fRWjaYpM23vrU5 UsMPyPGjTWsCqECBiFj7hX3kB0o763RuJGvUQWpzh3EVo+Lf15Rr9MQmQraXa9zB kO6ribzphXjeOqbkXY2OV1x8+Dv8cu6OwI8mDh7rA6bYlpXN3t7siFMnHRsaK5Hg ky49IAih22V6lU8ojSBlAJQPaqJeh11mZuDvpqjswH/aY4DA9xcJbFprk/DhIl5A gmpMpEgkLNCJfQm9KS19YJ9CweMgflUfv0CR9LYNUoN307thwZKxZlU8TjRsZg8/ myyGcNwiMQxIVvOD5qIDvJbZMHJhvuSQIEiDpVawUkn5hyXbn7Fc0tk9PZ6ajPne MCXmkmgOf0JNLcwPePdHORmrYOhILosnVGvWwzW7CRIgBKMmjJmJdoukgg62hHqb wpxqRx4FTBd7wK+zMtX2SfBPiXxIu2sKUo+EPr2T6M0yWWlHyhW1XZhjK+wHitDh OHlpmygYOdGcyxzCLUCnuRXSpF7bPAWjm/zBA4YXjM3H5VMXxV4cKw1+CbjqbrG8 M1wrcBC+GMIUtuwrgAH7y5SEg8tyWSUeJHvqU5P1sztTBvMamm4=
    =t19C
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    *
  • From Johannes Truschnigg@21:1/5 to All on Wed May 14 22:30:02 2025
    XPost: linux.debian.maint.boot

    Hey Cyril and Gaudenz,

    thanks for your interest in my patch :) I had a number of friendly off-list exchanges with both Pascal Hambourg and Holger Wansing about its potential impact and timing, and I communicated to Holger that I would be willing to try for offical inclusion for trixie+1.

    For Debian 13, I plan to keep monkeypatching my personal installation media
    for where I need this fix/feature, just as I have been doing it with Debian
    12.

    I'd be happy to share my (somewhat ugly, but effective) method to do that for USB-based installation media, if so inquired via mail or IRC :)

    Cheers!

    --
    with best regards:
    - Johannes Truschnigg ( johannes@truschnigg.info )

    www: https://johannes.truschnigg.info/

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEEGu9IhkI+7/aKLUWF95W3jMsYfLUFAmgk+dcACgkQ95W3jMsY fLXlVw/8DUYNiOoKgHpuyFZG74+3ejHJ7xAGjcI60x+1OKpHtcgV4eYoVTSGmMrw gdBCEWFmpccPZPVRIbXPPYtypDGEuhhO2Bj/SK49fJ3p8FcGUH8dEcUCi30k3uxI +Ss6R7pm/X2yQPmzBroG3hvvHquPAwi95ajinwnBiAIGYIvK/1nXekfB8uO1dRiU VKGSy6ibkCnbiFN71/zlE8x33b6WztCvDeeMQf8pNHhwJOeZaXy1GYJdpeIUGboc QM6hwPMYu2a3GooXDseOWDJsdDWL8OM6BZSYQbVETaPPlula1QacQn6I4DGD2aJU uDAOdkZ7pgbeTZt3VUhgk9TFNSPZZfLBHSgXyXDek5bDSRk6JCHC/Vz3Hkpn1H53 axx2kVdKSiLXL7tmeUBGU3o68cOD6czJOz9tlfYanVpBXXtvx7EGL5WnJ7Ane4J3 fkb5rU3/Bm9bP8Md6NMbIotSnx+G8J3srcLBZvJ+2kLLAlckLRUdZnPNW2DJAiNN NPmPO+PDkhuO8FlladzjFy9i8K4Cs+mfpKnjucdsF8GHIBfTr5dbVCkIlnVKYfQt uaXU9UTQNYFwBQEwHF0I5YHrZ73PVQEVKlhbkFM9JJ/vgBtecs6CcwvOHcdBswIk 42RcZiFbrGshSxPozdp6vzssAtl3cr9+yf5oDRqRJb3GjdTq6FM=
    =aotj
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet
  • From Pascal Hambourg@21:1/5 to Cyril Brulebois on Fri May 16 10:10:01 2025
    XPost: linux.debian.maint.boot

    On 14/05/2025 at 21:47, Cyril Brulebois wrote:

    Feel free to add this to the wishlist, without any promises though.

    https://wiki.debian.org/DebianInstaller/Trixie/Wishlist

    I can do it if Johannes agrees.

    I haven't tried to fully understand the patches and its implications, but that might be doable even if we're getting late in the release cycle.

    Patch 1 (amended by patch 3) enforces that partman-auto-raid recipe
    separator periods are isolated (surrounded by blanks), so that dots
    which may exist in recipe elements (devices, extra-args...) are not
    processed as recipe separators any more. E.g.:

    1 2 0 fat32 - /dev/disk/by-path/pci-0000:61:00.0-nvme-1#/dev/disk/by-path/pci-0000:a4:00.0-nvme-1
    # --metadata=1.0 .

    The implication is that existing partman-auto-raid recipes in which a
    separator period is appended to the last recipe element will not be
    supported any more, e.g.:

    1 2 0 ext4 / /dev/sda1#/dev/sdb1.

    This is what is already enforced by partman-auto recipe parser and
    illustrated in all examples provided by partman-auto-raid documentation
    [1], even though it does not formally state that separator periods must
    be isolated.

    [1] <https://sources.debian.org/src/debian-installer/testing/doc/devel/partman-auto-raid-recipe.txt>

    Patch 2 adds support for the special "efi" method in partman-auto-raid
    recipes. However it seems to be incomplete, as IIUC it does not set the
    "esp" flag/partition type identifier on RAID member partitions, which
    may be required by some UEFI implementations to recognize such
    partitions as ESP when booting from the removable media path (other UEFI implementations may be able to use any partition containing a FAT
    filesystem as an ESP, regardless of the partition type identifier). Note
    that grub-install cannot update EFI boot variables if /boot/efi is not a
    plain partition, so booting from such setup relies on the removable
    media path or manually updating EFI boot variables with efibootmgr via
    preseed commands.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Johannes Truschnigg@21:1/5 to Pascal Hambourg on Sat May 17 11:40:01 2025
    XPost: linux.debian.maint.boot

    On Fri, May 16, 2025 at 10:05:50AM +0200, Pascal Hambourg wrote:
    On 14/05/2025 at 21:47, Cyril Brulebois wrote:

    Feel free to add this to the wishlist, without any promises though.

    https://wiki.debian.org/DebianInstaller/Trixie/Wishlist

    I can do it if Johannes agrees.

    Yeah sure, I personally would be fine with that. I don't want to step on Holger's toes after our off-list discussion of the matter, however... but I realize having made it onto the wishlist is very far from the final call to
    get it merged for a given release, so I hope he'd be fine with it ;)


    [...]
    Patch 2 adds support for the special "efi" method in partman-auto-raid recipes. However it seems to be incomplete, as IIUC it does not set the
    "esp" flag/partition type identifier on RAID member partitions, which may be required by some UEFI implementations to recognize such partitions as ESP when booting from the removable media path (other UEFI implementations may
    be able to use any partition containing a FAT filesystem as an ESP, regardless of the partition type identifier). Note that grub-install cannot update EFI boot variables if /boot/efi is not a plain partition, so booting from such setup relies on the removable media path or manually updating EFI boot variables with efibootmgr via preseed commands.

    Hrrmm, interesting - I did assume that the "method" := "efi" approach that my patch uses would cause the resulting partition to have the proper metadata to be universally recognized as an ESP, but at least parted 3.5 is not convinced: I just checked the result of one of my pre-seeded, patched Debian 12 installs with that ESP-on-RAID1-stuff, and while the system does boot without problems (relying on the removable-media support, as you wrote), the "boot, esp" flags are missing from parted's "Flags" output...


    I am not sure what (if anything) is at fault here, and won't have enough time today to go into the details, but looking at hexdumps from two legs of a resulting RAID1 and my own desktop's single-m.2.EFI setup, the crucial bit
    (ESP GUID) seems to be OK? (I am not a EFI expert, unfortunately, so I am very open to hearing your corrections!)

    First RAID1 leg (sda):
    --- >8 ---
    00000200 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b |(s*......K...>.;| 00000210 0a 58 6e 66 16 d6 57 43 8b 45 76 3c f5 18 7d d8 |.Xnf..WC.Ev<..}.| 00000220 00 08 00 00 00 00 00 00 ff 3f 0f 00 00 00 00 00 |.........?......| 00000230 00 00 00 00 00 00 00 00 45 00 53 00 50 00 72 00 |........E.S.P.r.| 00000240 61 00 69 00 64 00 00 00 00 00 00 00 00 00 00 00 |a.i.d...........| 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| --- 8< ---

    Second RAID1 leg (sdb):
    --- >8 ---
    00000200 28 73 2a c1 1f f8 d2 11 ba 4b 00 a0 c9 3e c9 3b |(s*......K...>.;| 00000210 2b a5 26 4f 2a 9f 0c 48 95 81 10 b0 a6 40 81 03 |+.&O*..H.....@..| 00000220 00 08 00 00 00 00 00 00 ff 3f 0f 00 00 00 00 00 |.........?......| 00000230 00 00 00 00 00 00 00 00 45 00 53 00 50 00 72 00 |........E.S.P.r.| 00000240 61 00 69 00 64 00 00 00 00 00 00 00 00 00 00 00 |a.i.d...........| 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| -
  • From Holger Wansing@21:1/5 to All on Sat May 17 12:30:01 2025
    XPost: linux.debian.maint.boot

    Hi,

    Am 17. Mai 2025 11:28:45 MESZ schrieb Johannes Truschnigg <johannes@truschnigg.info>:
    Yeah sure, I personally would be fine with that. I don't want to step on >Holger's toes after our off-list discussion of the matter, however... but I >realize having made it onto the wishlist is very far from the final call to >get it merged for a given release, so I hope he'd be fine with it ;)

    I'm fine with whatever you decide. No worries.


    Holger



    --
    Sent from /e/ OS on Fairphone3

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Pascal Hambourg@21:1/5 to Johannes Truschnigg on Sat May 17 21:00:01 2025
    XPost: linux.debian.maint.boot

    (Trimming the Cc list, no need to bother everyone)

    On 17/05/2025 at 11:28, Johannes Truschnigg wrote:
    On Fri, May 16, 2025 at 10:05:50AM +0200, Pascal Hambourg wrote:

    Patch 2 adds support for the special "efi" method in partman-auto-raid
    recipes. However it seems to be incomplete, as IIUC it does not set the
    "esp" flag/partition type identifier on RAID member partitions (...)

    Hrrmm, interesting - I did assume that the "method" := "efi" approach that my patch uses would cause the resulting partition to have the proper metadata to be universally recognized as an ESP

    method=efi would only make partman try to set the "esp" flag
    (representing the ESP type identifier) on the resulting RAID array
    /dev/mdX, not on member partitions. "boot" is only a duplicate of "esp"
    on GPT.

    but at least parted 3.5 is not convinced: (...) the "boot, esp" flags
    are missing from parted's "Flags" output...

    I am not sure what (if anything) is at fault here, and won't have enough time today to go into the details, but looking at hexdumps from two legs of a resulting RAID1 and my own desktop's single-m.2.EFI setup, the crucial bit (ESP GUID) seems to be OK? (I am not a EFI expert, unfortunately, so I am very
    open to hearing your corrections!)

    I would trust more parted -l or fdisk -l output.

    PS: Any opinion about my latest comment ("nitpick") in the MR ?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Johannes Truschnigg@21:1/5 to Pascal Hambourg on Mon May 19 08:40:01 2025
    XPost: linux.debian.maint.boot

    On Sat, May 17, 2025 at 08:55:45PM +0200, Pascal Hambourg wrote:
    (Trimming the Cc list, no need to bother everyone)

    Very thoughtful, thanks! :)


    [...]
    Hrrmm, interesting - I did assume that the "method" := "efi" approach that my
    patch uses would cause the resulting partition to have the proper metadata to
    be universally recognized as an ESP

    method=efi would only make partman try to set the "esp" flag (representing the ESP type identifier) on the resulting RAID array /dev/mdX, not on member partitions. "boot" is only a duplicate of "esp" on GPT.

    Yeah, I can see now how this makes sense, and I guess my mind was playing a nasty trick on me and wrongly assumed that this would somehow affect the partition metadata, too. Thinking about it now, that cannot work, as that part of the block range of the disk (where partition metadata resides) is outside
    of the range that the md array itself cares about, so writing to the assembled array device could never ever touch that data.


    [...]
    I am not sure what (if anything) is at fault here, and won't have enough time
    today to go into the details, but looking at hexdumps from two legs of a resulting RAID1 and my own desktop's single-m.2.EFI setup, the crucial bit (ESP GUID) seems to be OK? (I am not a EFI expert, unfortunately, so I am very
    open to hearing your corrections!)

    I would trust more parted -l or fdisk -l output.

    That doesn't change the picture, they still identify the partition as "Linux RAID" - and that seems to be because of the Type-UUID, as displayed by fdisk
    in expert mode.

    A proper ESP has Type-UUID of "C12A7328-F81F-11D2-BA4B-00A0C93EC93B", while these md partitions pop out of d-i as "A19D880F-05FC-4D3B-A006-743F0F84911E" (Linux RAID).

    I just checked what happens if I use fdisk in expert mode to coerce a
    Type-UUID of "C12A7328-..." (ESP) for the partitions holding both RAID1 legs instead: That change keeps the md RAID1 array assembling and working fine, fdisk and parted agreeing upon the nature of the partition ("ESP", as I want
    it to), and is, I think, what the result of this exercise SHOULD be.

    md, for all I know, identifies candidate disks not by *partition* UUIDs, but
    by scanning LBA ranges of block devices for its own metadata, and that still holds when the Type-UUID has been altered to that of an ESP.

    So I guess if this is to be included in Debian proper, I will have to make
    sure that the RAID1 legs' partitions' Type-UUIDs will have to be bent to match the expected value for an ESP. What do you think, does that sound sane?


    PS: Any opinion about my latest comment ("nitpick") in the MR ?

    Yeah, that is perfectly reasonable - I will update the MR as soon as I can :)

    --
    with best regards:
    - Johannes Truschnigg ( johannes@truschnigg.info )

    www: https://johannes.truschnigg.info/

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEEGu9IhkI+7/aKLUWF95W3jMsYfLUFAmgq0UIACgkQ95W3jMsY fLVCPRAAreS1WUWgVybigDl5VF9l/FoEEwMMZJorHRJaw8xyezCf8XGRA81mrz6e mqOz8wUWjgwVog17nu8Btg2La6nCzTVtg/1LxUk1Z9FtC2ep5/icGu2/RVQ3yh++ eiR6iB2tMYeWa/TVHpBf8TJf94lQqWig4qx70LpBQvttkZqvv3aEvV4gJUwygBka wBVurM1sQEhJZDBQsafCej/mBoPSN0UhMYSqiS64dTvi85eqSXvIHZrUCQHHiwRF hrZWowD2+ZLnqEOkiFpQeN+iCFCwezkoqhVOjMpRi4xIC2WBQ1ETihvh6ov3Io0V d+1WUUotazMYlJHrL2cKq5grcLxy0Pn/E3YE5PXeDhH5ZQDjJPQx9mNHnfoMAnK0 RENzOcDZ7uSRJv7P3nhlDHFpwGHrDQOGPT3iI+BVCd+Yg96J5GE4t7QVlB29YTA+ A0Z/L7tET4sWvKVKlNCvJmDzdrsf+XxHTIoIlfjrfkJtbNArJKMSAWGPgAK2fvN5 G0f6D94zEpcK9cM1tJKZTzTIjt72LqOQItJRfautWI2zNEl7b3PsQdKRJEo9azsj xpF1lyivjzSI4Llz6cs24g29LCgp+Dc9OeXqYAoKjXw0LCiXs47FrwQgFVwgfYQW xDNhBSiqsiwrVosCNpL+0D8iqY4cjACqUhW63JTwnCJEQvSEPzw=
    =+Lv2
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet