LVM thin-provisioned volumes

[jpouellet]

It is my understanding that Qubes plans to moves to LVM thin provisioning by default in R4.0 (QubesOS/qubes-issues#2412), and this does not appear to be supported by Heads at the moment:

[osresearch]

Can you try turning it on in the config/linux.config file and seeing if it works?

diff --git a/config/linux.config b/config/linux.config
index 7ad8bab..14cc7f8 100644
--- a/config/linux.config
+++ b/config/linux.config
@@ -946,9 +946,11 @@ CONFIG_BLK_DEV_DM=y
 # CONFIG_DM_DEBUG is not set
 CONFIG_DM_BUFIO=y
 # CONFIG_DM_DEBUG_BLOCK_STACK_TRACING is not set
+CONFIG_DM_BIO_PRISON=y
+CONFIG_DM_PERSISTENT_DATA=y
 CONFIG_DM_CRYPT=y
 # CONFIG_DM_SNAPSHOT is not set
-# CONFIG_DM_THIN_PROVISIONING is not set
+CONFIG_DM_THIN_PROVISIONING=y
 # CONFIG_DM_CACHE is not set
 # CONFIG_DM_ERA is not set
 # CONFIG_DM_MIRROR is not set
@@ -2496,7 +2498,7 @@ CONFIG_CRC32_SLICEBY8=y
 # CONFIG_CRC32_SARWATE is not set
 # CONFIG_CRC32_BIT is not set
 CONFIG_CRC7=m
-CONFIG_LIBCRC32C=m
+CONFIG_LIBCRC32C=y
 CONFIG_CRC8=m
 # CONFIG_AUDIT_ARCH_COMPAT_GENERIC is not set
 # CONFIG_RANDOM32_SELFTEST is not set

[osresearch]

(unrelated to the LVM topic -- are you running Xen 4.6.1?)

[jpouellet]

Can you try turning it on in the config/linux.config file and seeing if it works?

After a while of rebuilding & reflashing Heads, and reinstalling Qubes with thin partitioning, same error. However, I didn't make clean, and none of the usual ways to read config of the running kernel seem to work, so it may be possible that I'm still running the old kernel? Rebuilding again to see if that might have been it... :/

For me to say that things actually work though, the documentation would need a bit more love. The intended qubes disk layout is unclear to me, and I've still not yet been able to successfully boot Qubes even once.

/bin/qubes-boot expects a /dev/qubes_dom0/00, but a default install only produces /dev/qubes_dom0/{root,swap}. However, if you select custom partitioning in the installer, the swap volume does get named 00. So you have some extra unnamed LV which you're calling root, but isn't root? I'm confused.

If I modify the script to get the blkid for the "root" LV instead of 00 (and use /bin/xen.gz instead of the one in /boot because otherwise I only ever get VGA rainbows) then I can kexec into the kernel & initramfs installed to /boot by qubes, which gives me a cryptsetup prompt and then sits at "Reached target Basic System." for several minutes before dropping me into a dracut recovery shell:

at which point I'm lost. (and rdsosreport.txt is an empty file)

I'm first trying to even see if I can even get Qubes to boot at all, nevermind the TPM and dm-verity... So far I've only been able to successfully boot the installer (via /bin/qubes-install).

(unrelated to the LVM topic -- are you running Xen 4.6.1?)

No, that's just what ships on the Qubes R3.2 install ISO. This is a dedicated machine I keep reinstalling.

[jpouellet]

Okay, so I'm an idiot. I had only rebuilt qemu.img, haha.

New failure mode:

Also, zooming in on https://www.flickr.com/photos/osr/33156102504/ is informative re: your partitioning scheme. Documentation in disguise :)

[jpouellet]

Okay, enough reinstalling for today. I'm going to go back to normal LVM and try to get this thing booting. Future tests should be possible with qemu.

[jpouellet]

In other (non-LVM-related) news, with:

• / on 00
• swap on 01
• /home on 02

Qubes boots! :)

[osresearch]

Finally getting around to installing a Qubes 4.0-rc1 after SHA and I think I see the problem...

With the old partitioning setup each device was its own luks entry, while with the thin provisioning there is only one partition and it is encrypted. To get to the volume group info the device needs to be mapped:

cryptsetup luksOpen /dev/sda2 qubes_dom0

Now lvm lvscan will show /dev/qubes_dom0/root (and swap and home).

however, we're missing /usr/bin/thin_check (from https://github.com/jthornber/thin-provisioning-tools) so other things are breaking.

[osresearch]

And the kexec-save-key script fails as well, since there is only one encrypted volume and it is trying to get the LUKS header for each partition in the volume group.

[flammit]

I've got another x230 coming arriving this week that I was planning to dedicate to Qubes 4/Heads testing. Will check out the LVM thin provisioning over the weekend.

[osresearch]

@jpouellet were you able to install Qubes 4 under Heads? I'm getting an error during the post install steps (split into issue #237)