XPost: alt.os.linux, alt.windows7.general

A reminder, this is a Linux/Windows ngs cross-post, because it has
relevance to both. If you don't like that, please just stay out of the
thread rather than carping about it.

For those of you who remember the original thread begun in these same
ngs in June last year, I've now solved everything that most urgently
needed to be solved, except one newly discovered problem ...

For some strange reason, when booting directly from the USB the Grub
menu sometimes doesn't display, however the Grub menu on the hard disk
of the same PC always displays. Accordingly I copied the modules from
the subdirectories of /boot/grub on the hard disk onto the USB, and did
a diff between both grub.cfg files, and there are no significant
differences between their preambles. So officially I'm now baffled
about this.

In case it is helpful to others, here in full detail is what I have done
and some notes on how I did it ...

I created a 256GB USB with partitions as follows:

1 8GB FAT32 Win PE + Ghost + Product Key Scanner
2 8GB FAT32 Clonezilla
3 32GB Ext4 XUbuntu 22
4 Rest NTFS Ghost Image & Installation ISO Files.

Grub is installed, with its code in the XUbuntu partition, and the
options, which all boot successfully, are as follows (the Linux ones are abbreviated because, apart from some display string tidying up, they are
pretty much as installed by Grub, but for the rest the detail is
important, so they are given mostly in full:

menuentry 'XUbuntu 22' ... {
...
}
submenu 'Advanced options for XUbuntu 22' ... {
menuentry ... {
...
}
...
}

### BEGIN /etc/grub.d/20_memtest86+ ###
menuentry 'Memory test (memtest86+.elf)' {
...
}
menuentry 'Memory test (memtest86+.bin, serial console)' {
...
}
### BEGIN /etc/grub.d/40_custom ###

# See Note 1
menuentry 'Windows 32-Bit PE Environment With Ghost' --class windows
--class os {
savedefault
insmod part_msdos
insmod fat
set root='hd0,msdos1'
chainloader +1
}

# See Note 2
menuentry 'Clonezilla 64-Bit' {
savedefault
set root=(hd0,msdos2)
chainloader +1
}

# See Note 3
menuentry "Windows 98 Emergency Console" --unrestricted --class Windows {
savedefault
linux16 (hd0,msdos4)/ISOs/memdisk
initrd16 (hd0,msdos4)/ISOs/W98_Emergency_Boot_Disk.img
}

# See also Note 3
menuentry "Windows 98 Emergency Console With Dell Diagnostics & Ghost
v2003" --unrestricted --class Windows {
savedefault
linux16 (hd0,msdos4)/ISOs/memdisk
initrd16 (hd0,msdos4)/ISOs/W98_Emergency_Dell_Ghost.img
}

# See Note 4
menuentry "Ubuntu 18 64-Bit Desktop Installation" --unrestricted --class Windows {
savedefault
insmod ext2
insmod ntfs
set iso_path="/ISOs/ubuntu-18.04.1-desktop-amd64.iso"
loopback loop (hd0,msdos4)$iso_path
linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$iso_path quiet noeject noprompt splash
initrd (loop)/casper/initrd.lz
}

# Similarly for Ubuntu 22 & 24, see also Note 4
menuentry "Ubuntu 20 64-Bit Desktop Installation" --unrestricted --class Windows {
savedefault
insmod ext2
insmod ntfs
set iso_path="/ISOs/ubuntu-20.04.1-desktop-amd64.iso"
loopback loop (hd0,msdos4)$iso_path
linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$iso_path quiet noeject noprompt splash
initrd (loop)/casper/initrd
}

menuentry "Ubuntu 22 64-Bit Desktop Installation" --unrestricted --class Windows {
...
}

menuentry "Ubuntu 24 64-Bit Desktop Installation" --unrestricted --class Windows {
...
}

# Similarly for Windows 8 through 11, see Note 5
menuentry "Windows 7 Professional 64-Bit Install" --unrestricted --class Windows {
savedefault
set iso_path="(hd0,msdos4)/ISOs/Windows_07_Professional_64Bit_x17-59186.iso"
loopback loop $iso_path
linux16 (hd0,msdos4)/ISOs/wimboot
initrd16 \
newc:bootmgr:(loop)/bootmgr \
newc:BCD:(loop)/boot/bcd \
newc:boot.sdi:(loop)/boot/boot.sdi \
newc:boot.wim:(loop)/sources/boot.wim
}

menuentry "Windows 8.1 Professional 64-Bit Install" --unrestricted
--class Windows {
...
}

menuentry "Windows 10 Professional 64-Bit Install" --unrestricted
--class Windows {
...
}

menuentry "Windows 11 Professional 64-Bit Install" --unrestricted
--class Windows {
...
}
### END /etc/grub.d/40_custom ###


Note 1
======

A customised Win Pre-execution Environment (WinPE) may be made by
installing the Microsoft Windows Assessment and Deployment Kit (ADK) and
its associated PE environment (= 2 installations). Once the software
has been installed, you need to mount an architecture-dependent version
of the supplied image and apply updates to it, then create a working set
from the result, then apply customisations to the working set.
Instructions for this are beyond the scope of these notes, but may be
found here:

https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/winpe-create-usb-bootable-drive

My customised WinPE works very well for running Ghost, etc, but suffers
from one fault, in that its timezone is wrong, and the Ghost image files created have datetime stamps that are 8 hours ahead of UK time; I think
I know more or less how to fix this, but have yet to get around to doing so.

Curiously, a customised WinPE made as above is installed on a FAT32
partition, but those on the original Windows installation media are on
NTFS partitions.


Note 2
======

Readers of the original thread will remember that there was a subthread
about trying to get Clonezilla to boot off a hard disk or USB partition
in exactly the same way as it does when a USB is made directly from the
ISO, and that eventually I found a solution. Here's what I wrote
before, slightly updated:

WRT Clonezilla 3.1.2-22 and getting it to boot from a HD exactly as it
does from a USB, Clonezilla themselves don't actually tell you how to do
this ...
https://clonezilla.org/livehd.php
... the method they give boots not into the initial menu, but straight
into the first choice from that menu. However, Clonezilla is a syslinux system, and there are other sources of information about syslinux that
are more helpful, for example this, the relevance of which will become apparent:
https://wiki.archlinux.org/title/syslinux

WARNING: The following overwrites both the MBR of the entire disk and
the PBR of Clonezilla's partition, so if you have other bootable
partitions &/or grub installed to choose between them, all this will be
lost, so make any backups necessary for restoration before running the following command!

As long as you don't mind wrecking your multi-boot system and rebuilding
it afterwards, which may be why Clonezilla don't tell you about this
option, you can run a program to get it to boot off a HD exactly as it
does off a USB. In both USB and HD copies, there is a bash script ...
/utils/linux/makeboot.sh
... which can be run to make the installation bootable in exactly the
same way from either medium. Probably it would be needed for a USB only rarely, because programs that write the ISO image to the USB seem
generally to take care of things pretty well, but for a HD installation,
it seems to be the easiest way to get everything to work as it should.

However, if your Clonezilla HD partition is formatted FAT32 like a USB,
then you can't just run the file, because FAT32 doesn't support the x attribute, and by default modern linux systems tend to mount FAT32
partitions translating the attributes of all files as non-executable.
The easiest way around this is to run the script through bash ...
sudo bash <mounted path>/utils/linux/makeboot.sh /dev/sdxy
... where sdxy is the designation of the Clonezilla disk and partition,
in my case ...
sdb2

Now when you boot from that hard disk, it will boot straight into the
normal brightly coloured Clonezilla menu with large fonts, and will
behave seemingly exactly as if it had been booted off USB. Then you can reinstall grub to get everything else back. For the Clonezilla option, preserving what has been won, add to the file (in Ubuntu, other distros
may differ) ...
/etc/grub.d/40_custom
... the following ...
menuentry {
savedefault # If you've enabled saving the last grub choice
set root=(hdx,y) # Where x & y are the CZ disk and partition #s
chainloader +1
}


Note 3
======

Booting W9x images requires memdisk from the Clonezilla USB.

The first image is a floppy, the second a USB stick, recreated since the
first thread in June. Readers of that original thread may remember that getting that W98 Emergency USB stick image to boot was the most
difficult problem to solve, and that originally I never got it to boot
directly from Grub, but I have now solved this by remaking the image
completely virtually, as described below.

I'm pretty sure I know now what the original problem was, though I have
not done a definitive test to prove my theory, being content merely to
have solved the problem somehow or other. I think the problem was that
the original USB stick had a Microsoft MBR, which for a long time had a
bug which Microsoft eventually only 'solved' by obviating it through
adopting UEFI booting methods. The bug was this: no matter what number
disk the MBR was loaded from, it looked for the partition table from
Disk #0, not that of the disk from which it itself was loaded. The
definitive test would be to replace the MBR in the problem image, and
see what happens, but I'm afraid that, having solved the problem now by
other means, I can't be arsed.

Here's how I recreated the image in such a manner that it worked:

3.1 First, obtain a decent MBR. Historically, I've found those
installed by Lilo inter-work well with Microsoft's idiosyncrasies, but
Lilo is no longer available from recent versions of Linux, so here's a
way to obtain it (others more knowledgeable about Lilo &/or distribution
file arrangements may be able to suggest a way to download it directly):
3.1.1 Boot from an Ubuntu 18 USB, and choose to Test, NOT install.
3.1.2 Run a terminal
3.1.3 sudo -i
3.1.4 apt -y install lilo
(ignore the warning, has no relevance here)
3.1.5 Attach either a "don't care" USB stick or a USB drive bay
with a "don't care" disk in it; let us suppose that in this instance it
comes up as /dev/sdx
3.1.6 lilo -M /dev/sdx mbr
3.1.7 dd if=/dev/sdx of=<path>/Lilo_MBR.img bs=1 count=446
(<path> must be to some place that will survive exiting the Ubuntu 18 installation environment, say on a local HD which you will have to mount beforehand. Note that 446 stops just short of the disk's partition
table, you don't want to be copying that.)
3.1.8 Close down the Ubuntu 18 installation environment and reboot
into normal Linux.

3.2 Copy the MBR obtained as above to become the start of the new W98
system disk, then complete the first sector by creating an empty
partition table (64 bytes more) and appending the boot record tag (2
bytes more):

cp -a Lilo_MBR.img W98_Disk.img
dd if=/dev/zero of=W98_Disk.img bs=1 seek=446 count=64
echo -ne '\x55\xAA' | dd of=W98_Disk.img bs=1 seek=510

3.3 Extend the image to be large enough to contain what is required to
be on the disk, in my case a little under 50MB, so ...
(50*1024^2)/512 - 1 = 102399
... sectors

dd if=/dev/zero of=W98_Disk.img seek=1 count=102399

3.4 Create a partition table on the disk with a single FAT32 (LBA)
partition:

fdisk W98_Disk.img
n (accept subsequent defaults offered to create single partition
over whole disk)
t then 0c (change partition type to W95 FAT32 (LBA)
a (set sole partition active)
w (write the result back to the image and exit)

3.5 Make a FAT32 filesystem on the partition (if /dev/loop1 is already
in use, choose a successively higher number until a free one is found).
The offset given, 1048576, is that of the first partition:

losetup /dev/loop1 W98_Disk.img -o 1048576
mkdosfs -F32 -f 2 /dev/loop1
losetup -d /dev/loop1

3.6 Now there's the problem of installing the Microsoft W98 system.
It's important to understand that this includes not just the three
system files - which could be put in place simply by copying them in
the correct order: IO.SYS, MSDOS.SYS, COMMAND.COM - but also the
Partition Boot Record (PBR), the first sector of the partition, which
contains the boot code to load IO.SYS, and it is this latter which is
the problem. In my case, I already have a bootable W98 system floppy
disk image, and QEmu is installed, so I did it in the latter, otherwise
some method similar to obtaining the Lilo MBR described above will be
required (beware line-wrap, QEmu command should be all one line):

qemu-system-i386 -fda W98_Boot_Disk.img -hda W98_Disk.img -boot order=ab
SYS C: (sets up PBR and transfers the 3 system files - IO.SYS, MSDOS.SYS, COMMAND.COM - to the root directory in the correct order)

Then exit QEmu.

3.7 For the rest of the files, the image file can be mounted as a loop
device and the remaining files required for it can be copied in by any
normal procedure.


Note 4
======

The 'insmod ntfs' line is there because the ISOs are stored on an NTFS partition. Note that the Grub settings for the Ubuntu installation ISOs
are nearly identical, the only variation being for the initrd line ...

Ubuntu 18:
initrd (loop)/casper/initrd.lz # <-- .lz file extension

Ubuntu 20+:
initrd (loop)/casper/initrd # <-- No file extension


Note 5
======

For booting directly from Windows ISOs WimBoot is needed ...

https://github.com/ipxe/wimboot/releases

... which I have placed in the ISOs directory of the Image partition
(#4) on my USB stick. Then as described above and as found online I
added the following fragment to ...
/etc/grub.d/40_custom
... and it and all the installation ISOs for Windows 8 through 11 work.
I would imagine that Vista probably also would work, but that XP and
earlier would need a different arrangement:

menuentry "Windows 7 Professional 64-Bit Install" --unrestricted --class Windows {
savedefault
set iso_path="(hd0,msdos4)/ISOs/Windows_07_Professional_64Bit_x17-59186.iso"
loopback loop "$iso_path"
linux16 (hd0,msdos4)/ISOs/wimboot
initrd16 \
newc:bootmgr:(loop)/bootmgr \
newc:BCD:(loop)/boot/bcd \
newc:boot.sdi:(loop)/boot/boot.sdi \
newc:boot.wim:(loop)/sources/boot.wim
}

--

Fake news kills!

I may be contacted via the contact address given on my website:
www.macfh.co.uk

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)