openSUSE MicroOS

Managing System Extensions with sysextmgrcli

2026-04-23T15:27:00+00:00

Managing System Extensions on openSUSE MicroOS with `sysextmgrcli`

If you are running openSUSE MicroOS, you already know the drill: the root filesystem is read-only, and transactional updates are the law of the land. But what happens when you need to add software or system extensions without rebooting or messing with your base OS layers? E.g. You need strace or gdb to debug a running application, but a reboot to install this tools would change the situation.

Enter System Extensions (sysext images) and the utility designed to make them manageable: sysextmgrcli.

What is `sysextmgrcli`?

At its core, sysextmgrcli is a command-line client for managing systemd-sysext images and has been written by Thorsten Kukuk. It is designed specifically to play nice with the atomic nature of MicroOS.

Instead of forcing you to use sudo for every query, it talks to a background daemon (sysextmgrd) via Varlink. This architecture allows unprivileged users to list existing system extension images without needing root permissions, while the daemon handles the heavy lifting of downloads and verification via systemd-pull. For security reasons root provileges are still required for installing or updating sysext images.

The Architecture: Smart Snapshots

One of the cleverest things about sysextmgrcli is how it handles storage to be efficient and “rollback-safe”:

/var/lib/sysext-store: This is where the actual image files live. Since /var is a separate subvolume shared across all Btrfs snapshots, you only store the image once, saving disk space. If you have no network available, that’s the location for storing offline or even own build sysext images via e.g. an USB device.
/etc/extensions: This directory contains symlinks to the images in the store. Because /etc is part of your root snapshot, the extensions are tied to your current system state.

Why does this matter? If you perform a system rollback, your symlinks roll back too. This ensures the active sysext images always match the OS version you are currently booted into.

Essential Commands

Getting started is straightforward. Here are the primary commands you’ll use to manage your extensions:

1. Listing and Checking Images

Want to see what’s available or if your images are compatible with your current OS version?

# List all images and report compatibility
sysextmgrcli list

# Check for updates and verify compatibility
sysextmgrcli check

2. Installing New Extensions

You can install by providing a name and a source URL. The tool automatically handles SHA256 verification and checks if it fits your OS.

# --url is optional (default: https://download.opensuse.org/tumbleweed/appliances/ )
sysextmgrcli install [NAME] --url [https://your-image-repo.com](https://your-image-repo.com)

3. Maintenance and Updates

Updates are handled by comparing local files against remote manifests. If a newer version matches your current snapshot, it gets pulled down and symlinked.

# Update existing images to the latest compatible versions
sysextmgrcli update

# Clean up: Remove images in the store that are no longer referenced by any snapshot
sysextmgrcli cleanup

The “Activation” Catch

It is important to note that sysextmgrcli is a manager, not an activator. It handles the logistics: downloading, version checking, and symlinking. To actually “plug in” the extensions to your running system, you still use standard systemd-sysext commands:

Manual activation: systemd-sysext merge
Manual deactivation: systemd-sysext unmerge
Enable at boot: systemctl enable systemd-sysext.service

Available default system extention (sysext) images:

debug (babeltrace, gdb, ltrace, strace, traceroute)
gcc (cpp, gcc, make, patch)
git (git, git-core)

Summary

You need `git` on your openSUSE MicroOS ?

Just call sysextmgrcli install git ; systemd-sysext merge and use it…

You do not need ‘git’ anymore on your system ?

Just call systemd-sysext unmerge and it is not available anymore…

sysextmgrcli bridges the gap between static immutable infrastructure and the need for flexible system additions. By leveraging the Btrfs directory structure of MicroOS, it ensures your system remains clean, version-synced, and easy to manage.

YaST2 support for Full Disk Encryption with TPM2

2025-08-11T12:07:00+00:00

Intro

The openSUSE MicroOS installer (YaST2) supports since snapshot 20250809 Full Disk Encryption (FDE) secured by a TPM2 chip and measured boot or a FIDO2 key.

Since quite some time the openSUSE project provides already images with Full Disk Encryption and TPM2 (e.g. openSUSE-MicroOS.x86_64-kvm-and-xen-sdboot.qcow2 or openSUSE-MicroOS.x86_64-kvm-and-xen-grub-bls.qcow2 for virtualized environments and there is a bog describing how to setup such a system with YaST2.

Now the next step is done: YaST2 supports setting up FDE+TPM2 for openSUSE MicroOS, so this feature can be enabled in an easy way during installation.

This requires a bootloader following the Boot Loader Specification (BLS). Available are grub2-bls and systemd-boot. For openSUSE MicroOS we decided to make systemd-boot the default bootloader.

Hardware Requirement:

UEFI Firmware
TPM2 Chip or FIDO2 key which supports the hmac-secret extension
4GB Memory

Installation of openSUSE MicroOS

Boot installation media

Follow the workflow until “Installation Settings”
Installation Settings:
- Partitioning:
- Select “Guided Setup”, keep the defaults until the “Partitioning Scheme” screen.
- “Enable Disk Encryption” and enter the password. The password will be the recovery key.
- Accept changes
Finish Installation

Finish FDE Setup

Boot the new system

Re-enrollment

If the prediction system fails, a new policy must be created for the new measurements to replace the policy stored in the TPM2.

With the recovery PIN (the password for FDE):

  # sdbootutil --ask-pin update-predictions

Next Steps

The next step will be to make FDE+TPM2 the default for MicroOS.

Further Documentation

zypper single transaction mode now default

2025-07-29T08:48:00+00:00

Default zypper backend

Over the weekend we switched the default zypper backend from the classic_rpmtrans to the single_rpmtrans backend in openSUSE MicroOS.

Anyone who has followed the factory mailing list or openSUSE’s bugzilla over the last two years has probably noticed the regular discussions about faulty updates, mainly because libraries were updated too late. The result was that tools from %pre/%post scripts did not work and in the worst case made the system unbootable. Traditionally, zypper executes the rpm command separately for each operation in a transaction. The advantage is that with large updates and a small root partition, zypper can download the packages one after the other, making the update possible in the first place. One of the disadvantages, however, is that the update process is much slower with a large number of packages. And not only that, zypper also has to save the posttrans scripts and execute them manually at the end. This means that the update process runs differently than when all RPMs are updated in one with RPM. Therefore, the zypper developers have implemented a new backend that executes all operations in a single transaction with librpm. However, for various reasons this is not yet the standard and so there are still bug reports about faulty bootloader configurations or problems updating libopenssl when using hardware-optimized libraries.

How is this implemented?

On the command line and for systemd units, the environment variable ZYPP_SINGLE_RPMTRANS=1 is now set globally, which should cover most cases on MicroOS.

If there are still situations where zypper uses the old classic backend, please report this via bugzilla.

What does this mean for the users?

Hopefully: nothing

Or better: the update problems are gone. If you still see problems with packages, please report them via bugzilla so that we can fix them.

Presenting GRUB2 BLS

2025-07-18T15:00:00+00:00

Protecting against rogue devices in openSUSE with Full Disk Encryption

openSUSE have now multiple ways to configure a Full Disk Encryption (FDE) installation. A very secure and easy way (YaST2) of doing this is via user space tools, as we described multiple times (like here, here, or here). This solution is based on the systemd tool-set like systemd-cryptenroll, systemd-pcrlock and systemd-cryptsetup, among other, orchestrated by the in-house sdbootutil script.

One of the main advantages of using this systemd approach is the possibility of integrating multiple authentication methods. Together with the traditional password, asked at boot time during the initrd stage, we can now unlock the system using a certificate, a TPM2, or a FIDO2 key. We can mix some of them creating multiple LUKS2 key slots, and use, for example, a TPM2 to unlock the device in a unattended fashion and a FIDO2 key as a recovery mechanism.

Honestly, the TPM2, and the TPM2+PIN variation, are the most relevant ones for the user. As described in the other posts, the TPM2 is a (some times virtual) device that can attest the health of our system using a mechanism known as measured boot.

The tl; dr version of this is that each stage of the boot process, starting from the firmware, will load and “measure” the next stage before delegating the execution on it. For example, this means that there is a moment in the latest stages of the boot process where the UEFI firmware will load from the disk the boot loader into memory. This can be the shim, systemd-boot or grub2-bls. It will calculate a hash value (usually SHA256) and will command to the TPM2 an “extend” operation for one of the internal register (PCR).

The extension is a cryptographic computation that is very easy to calculate, but impossible to replicate. It is done to one of those internal registers (PCR) and consist of calculating the hash (again SHA256) of the old value of the PCR together with the hash of the component that we are measuring. This new value will replace the current PCR value, as is the only way to change those registers. The security property resides in that it is cryptographically impossible to force the write of a desired value on one of those PCRs, but very easy to calculate the final value.

So this means that if all the components of the boot chain process are measured (all the stages in the UEFI firmware, the firmware configuration, the boot loader, the command line, the kernel and even the initrd), the final PCRs values can be compared with our expectations, and discover if the system has been booted with a good known software and configuration, allowing us to instantly known if some component in the boot chain has been hacked or modified with out consent.

That is a powerful property to have, but what is more interesting is that we can have secrets that can only be open in case that we are in one of those good or recognized states. We can, for example, cipher (seal) the key that open an encrypted disk using the TPM2, together with a policy that will decipher (unseal) the same key only, and only if we are using the same TPM2 and the PCR values are on a list of expected ones. Those policies can be very complicated, and can include extra passwords, certificates or other checks that will be validated before the TPM2 can unseal the key.

With a mechanism like this in place, thanks to the systemd tools, we can now avoid entering the password to unlock the encrypted disk if the system is in a healthy state. Healthy in the sense that we cryptographically guarantee that the code and configurations used during the boot process are the expected one, and no one entered init=/bin/bash in our kernel command line, or replaced the kernel or initrd with a vulnerable one, for example.

With the integration that we made of this model in openSUSE, we can make updates of the system, including the boot loader or the kernel, and sdbootutil will transparently generate new predictions of expected PCR values that are now considered safe. This imply an update of the TPM2 policy, that will be taken into consideration for the next boot, so the automatic unlock will succeed. If something goes wrong and the expected PCR values are not meet, the user will need to enter the password that is stored in a different LUKS2 key slot to open the device, to audit the system and validate it.

The fault in the design

Using a TPM2 as described before is a clear increase in the security level, but it is not the final answer. Security is always asymptotic approximation.

Some years ago a physical attack was described for the Windows BitLocker FDE solution. BitLocker is also using the TPM2 in a similar way that was described before, but was not using encrypted session to communicate with the device. Intercepting the SPI bus was shown possible to recover the password that unlock the disk. systemd learned from that and used encrypted sessions early, but this attack can also be avoided if the policy used to unseal the key was also demanding a PIN or password that must be entered by the user. Now the TPM2 can only unseal the secret if the PCRs are in the correct state and the provided password is the correct one. Should be noted that AFAIK the SPI sniffing can work with Clevis.

But more recently a second attack was made public that fully affect the original proposal, and does not requires the sophistication of the original one. (Disclosure: the attack was also internally described independently months before and some counter measurements was put in place much early)

The article describes how that attack can be done checking in the initrd the filesystem UUID used to mount the encrypted device. This information is inside the /etc/crypttab stored in the initrd, that will do something like this:

systemd-cryptsetup attach cr_root /dev/disk/by-uuid/$UUID ‘none’ ‘tpm2-device=auto’

If the expected firmware, configuration files, kernel and initrd are used during the boot process then the TPM2’s PCRs registers will have values that match the policy that unlock the device and the sealed key can be now unsealed by the TPM2, the disk will be unlocked, the switch root will succeed and the boot process will continue in the rootfs.

But what if the original drive is replaced by one that has the same UUID (it is a public information after all) that is also encrypted? Then the PCRs will be in the same correct state. Note that in measure boot is the previous stage the one that measures the next one before delegating the execution. Then systemd-cryptsetup will try to use the TPM2 to unlock the device using the key successfully unsealed by the TPM2 and … will fail to open it, of course. The rogue device maybe have a TPM2 key slot in the LUKS2 header, but for sure cannot be open with this TPM2 nor with the secret password.

In this situation systemd-cryptsetup will ask for the password to unlock the device, and the attacker can enter one that this time will open the rogue device. The switch root will happen but now it will continue the boot process in the fake rootfs, and a program stored there can make questions to the TPM2, that still contains the good PCR values. One of the questions can be the unseal of the secret key using the current policy. And this time (as was done before), the TPM2 will agree to deliver the secret to the bad program. Game over.

There are solutions for this attack, of course.

One is again to use TPM2+PIN instead of TPM2, the same solution for the sniffing attack. In this case the first systemd-cryptsetup call will fail and a password will be asked to unlock the device. But now the bad program cannot ask to the TPM2 to unseal the device using the current policy. The PCR values will match, but the policy also requires the enter of a secret PIN or password known by the real user, and without it the unseal will fail and the key will be keep safe.

Another solution is somehow invalidate the policy, extending some of the PCRs involved before the switch root, so the policy cannot be applied anymore after that. This can be done automatically by systemd-cryptsetup using the measure-pcr=yes in /etc/crypttab. With this option PCR15 will be extended using the volume key, a secret that can only be extracted knowing some of the device keys. For this solution to work, PCR15 needs to be included in the current policy, with an expected value of 0x000..00, the default one. Once the rogue device is open by the hacker provided password, PCR15 will be automatically extended and the value will be different from 0x000..00, invalidating the policy before the switch root.

That is a good solution, but not for us. In the daily situation the user will need to update the system, and a new policy needs to be calculated to replace the old one (for example when the kernel is updated). Because with systemd-pcrlock the policy is stored in the TPM2 in one of the Non Volatile RAM slots (NVIndex), we need to protect it somehow, so it cannot be replaced by other process. For that systemd is storing a secret key (recovery PIN) in a different NVIndex that is sealed by the same policy! If the key cannot be automatically recovered, because the policy does not apply anymore, then the recovery PIN will be asked to the user, making the update process a bit unpleasant if the policy is always invalidated.

Finally, another way to address the issue is to stop the boot process if we detect that the device is not the expected one. We can think of a new service, living in initrd that is executed in the very last moment, just before the switch root, that can stop the boot process (maybe halting the system) if the device that stores the rootfs is not the expected one.

For this, PCR15 is still a good solution. It contains the measurement of a secret (volume key) that can only be known by the real user, and cannot be replicated by the attacker. Ideally we can create a prediction for PCR15 and make this service to compare the effective value with the expected one, and if they are different then it can stop the boot process.

This is what the measure-pcr-validator service from sdbootutil is doing. sdbootutil first generates a prediction for all the encrypted devices that are opened during the initrd, and check that the correct tag is present in /etc/crypttab. To be able to access the volume key, the tool needs the root password, so this prediction is only update when it is really necessary, like for example when a new encrypted device is added. This prediction is signed by a private key stored in the host, as an extra security measurement, but because the public key is also stored in the ESP it is honestly not adding too much.

An extra service (measure-pcr-generator) will put some order on how the encrypted devices are opened, as this order is critical to produce a single possible PCR15 value. If we have one single device the order of measurements is not relevant, but if when have three (rootfs, /home, and swap, for example) we can have six possible and valid different values for PCR15.

The last step is that the dracut-pcr-signature service in the initrd will import from the ESP the prediction, the signature and the public key, so measure-pcr-validator can check the signature and compare the PCR value.

And that is all!

This approach is also kind of similar to what the new systemd-validatefs is doing, but for a file system level.

Presenting GRUB2 BLS

2024-10-08T12:55:00+00:00

GRUB2 with BLS is now in MicroOS and Tumbleweed

Recently the openSUSE project released for MicroOS and Tumbleweed a new version of the GRUB2 package, with a new subpackage grub2-$ARCH-efi-bls. This subpackage deliver a new EFI file, grubbls.efi, that can be used as replacement of the traditional grub.efi.

The new PE binary is a version of GRUB2 that includes a set of patches from Fedora, that makes the bootloader follows the boot loader specification (BLS). This will make GRUB2 understand the boot entries from /boot/efi/loader/entries, and dynamically generate the boot menu showed during boot time.

This is really important for full disk encryption (FDE), because this means that now we can re-use all the architecture and tools designed for systemd-boot. For example, installing or updating the boot loader can now be done with sdbootutil install, the suse-module-tools scriptlets will create new BLS entries when a new kernel is installed, and the tukit and snapper plugins will take care of doing the right thing when snapshots are created or removed.

Reusing all those tools without change was a great win, but what is better is that some of the quirks that the classical GRUB2 presented when extending the event log are not present anymore. Before this package, sdbootutil needed to take ownership of the grub.conf file, as this will be measured by GRUB2 by executed lines. That is right, for each line that is read and executed by the GRUB2 parser, a new PCR#8 will take place, and because GRUB2 support conditional as other complex constructors, it is very hard to predict the final value of PCR#8 without imposing a very minimal and strict grub.conf.

But with the new BLS subpackage this file, together with the fonts and graphical assets for the theme, and the required modules (like bli.mod) are now included in the internal squashfs inside the EFI binary. GRUB2 will not measure those internal files, without decreasing the security guarantees, because is now the firmware the one that measures the full EFI when the bootloader is executed during the boot process.

As today we cannot use YaST2 to install GRUB2 with BLS, but we can do that manually very easily. We need to make a systemd-boot installation, replace LOADER_TYPE from systemd-boot to grub2-bls in /etc/sysconfig/bootloader, install the new GRUB2 BLS package, and do sdbootutil install. Another option is to play with one of the available images for MicroOS or Tumbleweed.

Quickstart in Full Disk Encryption with TPM and YaST2

2024-09-03T12:07:00+00:00

Intro

This is a quick start guide for Full Disk Encryption with TPM or FIDO2 and YaST2 on openSUSE MicroOS. It focuses on the few steps to install openSUSE MicroOS with YaST2 and using Full Disk Encryption secured by a TPM2 chip and measured boot or a FIDO2 key.

Hardware Requirement:

UEFI Firmware
TPM2 Chip or FIDO2 key which supports the hmac-secret extension
2GB Memory

Installation of openSUSE MicroOS

Boot installation media

To avoid the use of the YaST2 expert partitioner, the default LUKS key derivation function from YaST2 will be used and three unused subvolumes (/boot/grub2/i386-pc, /boot/grub2/x86_64-efi, /boot/writeable) will be created. If you are familiar with the YaST2 expert partitioner, you can remove the three btrfs subvolumes and change the LUKS key derivation function to argon2id already during installation.

Follow the workflow until “Installation Settings”
Installation Settings:
- Partitioning:
  - Select “Guided Setup” and “Enable Disk Encryption”, keep the other defaults
- Booting:
  - Change Boot Loader Type from “GRUB2 for EFI” to “Systemd Boot”, ignore “Systemd-boot support is work in progress” and continue
Finish Installation

The order of adjusting the partitioning and changing the bootloader is important.

Finish FDE Setup

Boot new system

Enter passphrase to unlock disk during boot
Login
Enroll system:
- With TPM2 chip: sdbootutil enroll --method tpm2
- With FIDO2 key: sdbootutil enroll --method fido2

Optional, but recommended:

Upgrade your LUKS key derivation function (do that for every encrypted device listed in /etc/crypttab):

      # cryptsetup luksConvertKey /dev/vdaX --pbkdf argon2id
      # cryptsetup luksConvertKey /dev/vdaY --pbkdf argon2id

Adjusting kernel boot parameters

The configuration file for kernel command line options is /etc/kernel/cmdline.

After editing this file, call sdbootutil update-all-entries to update the bootloader configuration. If that option does not exist yet or does not work, a workaround is: sdbootutil remove-all-kernels && sdbootutil add-all-kernels.

Re-enrollment

If the prediction system fails, a new policy must be created for the new measurements to replace the policy stored in the TPM2.

If you have a recovery PIN:

  # sdbootutil --ask-pin update-predictions

If you don’t have the recovery PIN, you can set one with this steps:

  # sdbootutil unenroll --method=tpm2
  # PIN=<new recovery PIN> sdbootutil enroll --method=tpm2

Virtual Machines

If your machine is a VM, it is recommended to remove the “0” from the FDE_SEAL_PCR_LIST variable in /etc/sysconfig/fde-tools. An update of the hypervisor can change PCR0. Since such an update is not visible inside the VM, the PCR values cannot be updated. As result, the disk cannot be decrypted automatically at the next boot, the recovery key needs to be entered and a manual re-enrollment is necessary.

Next Steps

The next steps will be:

Support grub2-BLS (grub2 following the Boot Loader Specification)
Add support to the installers (YaST2 and Agama)
Make this the default if a TPM2 chip is present

Any help is welcome!

Further Documentation

Configuration files in /etc and /usr/etc

2024-08-14T13:28:00+00:00

This blog post dates from December 2019 and was updated in August 2024.

Intro

As some may have already noticed, several configuration files are no longer in /etc but are below /usr. This can be /usr/etc or directories below /usr/lib or /usr/share, depending on upstream projects.

What’s behind this move? For a better understanding, let’s first look how configuration file updates are handled today:

RPM and Configuration Files

RPM has limited support for updating configuration files. In the end this consist of two simple choices:

modified configuration files are moved away during upgrade and the admin has to redo the changes (.rpmsave files).
modfied configuration files are kept and changes done by the distribution are ignored (.rpmnew files). In the end the service may not work or could even be insecure!

Both options are not really user friendly and will most likely lead to a broken or insecure service after an upgrade, which requires manual work by the admin. On desktop systems or a simple server this may be tolerable, but for big clusters this can lead to a huge amount of work.

There are several alternative solutions for this like Three-Way-Diff or doing the update interactively, but the first one does not solve the problem if conflicting changes are occur, and the second one is no solution for fully automated updates.

Atomic Updates

For atomic systems another layer of complexity is added, because different states may contain different versions of a configuration file. So how can this happen? An atomic update is a kind of update that:

Is atomic
- The update is either fully applied or not applied at all
- The update does not influence your running system
Can be rolled back
- If the update fails or if the update is not compatible, you can quickly restore the situation as it was before the update

The update will be activated by rebooting into the new state, so after an update, before the reboot, the changes done by the update are not visible. If an admin or configuration management software changes the configuration files in the runnung system during this time, this will create conflicts, and needs manual interaction again.

Image Based Updates

Image based updates here means the OS (mostly /usr) get’s updated as image, we don’t speak about raw disk images. Since you deploy a new image for e.g. /usr, there is no tool to update configuration files outside this image, e.g. in /etc. And the image cannot overwrite /etc, since this would mean all local host depending configuration will go lost. So a strict separation of distribution provided configuration file and admin made changes is necessary. This is also called hermetic-usr.

Goal

The goal is to provide a concept working for most packages and their configuration files, which makes automatic updates much easier and robust. For that a new way to store and manage configuration files is needed.

Requirements for a Solution

The new solution should make sure that:

It’s visible to the admin that something got updated
It’s visible which changes the admin made
Package and admin changes should be merged automatically
There should be only one directory to search for default configuration files

Solutions

As a longterm solution no package should install anything into /etc any more, this directory should only contain host specific configuration files created during installation and changes made by the system administrator. Packages are supposed to install their default configuration files to another directory instead.

For SUSE/openSUSE the decision was made to use /usr/etc as the directory for the distribution provided configuration files.

For merging the package and admin configuration files there will have to be different strategies depending on the file type; the files can be categorized as follows:

Configuration files for applications
Configuration files for the system (network, hardware, …)
“Databases” like files (/etc/rpc, /etc/services, /etc/protocols)
System and user accounts (/etc/passwd, /etc/group, /etc/shadow)

Application Configuration Files

For application configuration files there is already a good solution used by systemd, which could be adopted for most applications:

/usr/etc/app.conf is the distribution provided configuration file.
If it exists, /etc/app.conf replaces /usr/etc/app.conf.
/etc/app.conf.d/*.conf contains snippets overiding single entries from /usr/etc/app.conf or /etc/app.conf.

This is a formal Configuration Files Specification by the UAPI Group.

The workflow for the application to load the configuration file would be:

Application looks for /etc/app.conf.
If this file does not exist, load /usr/etc/app.conf.
Look for overides in /etc/app.conf.d and merge them.

See https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Examples, “Overriding vendor settings” for more details and examples. A C library which provides a simple interface and implements above loading mechanism transparently for the application is libeconf.

Depending on the configuration file format above patterns may not work for all applications. For those applications a solution following the above guidelines as closely as possible should be found.

System Configuration Files (network, hardware, …)

As these configuration files are system specific and only created during or after installation and not provided by the distribution, these files will stay in /etc.

System Databases (rpc, services, protocols)

There are files in /etc which, strictly speaking, are no configuration files, such as /etc/rpc, /etc/services and /etc/protocols. They are changed very rarely, but sometimes new system applications or third party software need to make additions. These files will be moved to /usr/etc; /etc/nsswitch.conf has to be changed to search in /etc first and in /usr/etc second. A glibc NSS plugin usrfiles will be used for this. /etc will contain only the changes done by the admin and third party software.

/etc/passwd, /etc/group and /etc/shadow

There is no solution yet for these configuration files which would really solve the problems. Ideas are welcome!

Further Documentation

The original, full proposal with many more ideas and background information on the reasoning behind the decisions: Atomic Updates and /etc
The openSUSE wiki page: Packaging /usr/etc
Formal Configuration Files Specification
UAPI Group tracking upstream projects not supporting hermetic-usr for configuration
Hermetic /usr, as explained in the blog Fitting Everything Together
Factory Reset

Introduction of soft-reboot

2024-06-13T12:55:00+00:00

Intro

systemd v254 introduced a new reboot method: systemctl soft-reboot. A soft-reboot is similar to a regular reboot, except it affects user-space only. This command does:

Shutdown of all running services and other units
Optional: switch to a new root file-system
Re-exec systemd
Start all systemd services again

The advantage of this method is a pretty fast reboot:

No hardware, firmware and bootloader
No kernel
No initrd

But there are some challenges to be solved:

Kernel tables will not be flushed (may break services like firewalld+podman).
No initrd means that everything that is normally done in the initrd is not executed. In the case of MicroOS, this will break mount of overlayfs for /etc with transactional-update or relabeling the system for a new SELinux policy.
/run will not be cleared, which means reboot needed trigger will not be removed. This may affect many more services not expecting that files survive in /run.

Nevertheless, this method is ideal for quick reboots with transactional-update, e.g. if you only installed another package.

Current status

Support for soft-reboot is fully integrated in transactional-update and rebootmgr. The known, reproducible problems with soft-reboot are solved.

Currently soft-reboot is still disabled in transactional-update, but will be enabled by default in the near future.

How to use

An admin can trigger an immediate soft-reboot with: systemctl soft-reboot
A scheduled soft-reboot via rebootmgr can be triggered with: rebootmgrctl soft-reboot
With transactional-update it is not possible to enforce a soft-reboot, if soft-reboot is enabled, tranactional-update reboot will choose the best fitting reboot method based on the list of package updates. See man zypp-boot-plugin for more details and how to configure this.

If a soft or a hard reboot was done can be checked with the last command. The entries s-reboot or soft-reboot indicates a soft-reboot was done, reboot means this was a normal reboot. The detection will only work reliable with systemd v256 or newer.

Installation and Configuration

A prerequisite for this feature is that the patterns-microos-base-zypper package is installed. This pattern may be missing on old installations that have only been updated regularly.

The zypper plugin zypp-boot-plugin analyses the list of packages which get installed or updated and defines, if a hard reboot is required or a soft-reboot is good enough. /usr/etc/zypp/zypp-boot-plugin.conf contains the default configuration, which can be overwritten with drop-ins in /etc/zypp/zypp-boot-plugin.conf.d/*.conf

The configuration file /usr/etc/tukit.conf defines the system default. The variable REBOOT_ALLOW_SOFT_REBOOT is either set to false, this means soft-reboot is by default disabled, or true, which means a hard reboot is always done if a reboot is requested.

How to enable

To enable soft-reboot support in transactional-update, create the directory /etc/tukit.conf.d/ and create a file:

echo "REBOOT_ALLOW_SOFT_REBOOT=true" > /etc/tukit.conf.d/soft-reboot.conf

How to disable

To disable soft-reboot support in transactional-update, create the directory /etc/tukit.conf.d/ and create a file:

echo "REBOOT_ALLOW_SOFT_REBOOT=false" > /etc/tukit.conf.d/soft-reboot.conf

Easily set up Passwordless SSH Authentication

2024-05-17T09:20:37+00:00

MicroOS and Tumbleweed gained a new feature: You can now easily enroll SSH keys for authentication through the firstboot wizard on the very first boot already!

Motivation

Public-key based authentication is safer than password authentication and in most cases also easier to use and maintain. By default, OpenSSH does not even allow root to authenticate with a password anymore.

Question is how to get the desired public keys into the system to be able to log in. ssh-copy-id needs to be able to log into the target system, which requires password auth. Transferring the keys manually is tedious and error-prone.

With ssh-pairing there’s a new approach: a dialog based workflow allows interactive key exchange in both directions. After making an ssh connection and accepting the enrollment, the server’s key is added to the known_hosts lists on the client and the selected public keys from the client added to the server’s authorized_keys file.

How does it look like?

After starting the pairing procedure, the dialog shows what to do:

After the connection has completed, keys to accept can be individually reviewed and selected.

Where can I get it?

Starting from Snapshot 20240507, MicroOS and Tumbleweed images include this feature in the firstboot wizard. If you want to make use of this on other systems, you can install the ssh-pairing package and run the command with the same name (as root).

What about other distros?

ssh-pairing is distro-independent, see the project page for installation and usage instructions.

It would be very welcome to have this available for other distros out of the box as well! If you have questions or find any issues, please don’t hestitate to open tickets.

Have a lot of fun!

Systemd-boot and Full Disk Encryption with TPM and FIDO2

2023-12-20T09:00:00+00:00

Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS

openSUSE Tumbleweed and MicroOS are now delivering an image that is using systemd-boot as boot loader and full disk encryption based also on systemd. The unlock of the encrypted device can be done via the traditional password, a TPM2 (a crypto-device that is already present in your system) that will attach the device if the system is in good health, or a FIDO2 key that will validate the ownership of a token.

There is a lot to explain here, but basically those changes are in the direction of moving the distribution into a more safe place. For one side is making the design of the distribution much more simple, and for another it is following the current trends about security that other distributions are also aligning with.

So, lets start with the beginning …

systemd-boot

We all know and love GRUB2. It is a good boot loader. It is also big, complex, rich, massive and tends to move slow on the development side.

The openSUSE package for this boot loader contains more than 200 patches. Some of those patches are there for the last 5, 6 … 10 years. That is both an indication of the talent of the maintainers, but also can signal an issue in how slow the upstream contribution process can be.

GRUB2 supports all the relevant systems, including mainframes, arm or powerpc. Multiple types of file systems, including btrfs or NTFS. It contains a full network stack, an USB stack, a terminal, can be scripted … In some sense, it is almost a mini OS by itself.

But then UEFI happened 18 years ago, making almost all the features provided by GRUB2 somehow redundant. The system firmware was already providing most of these functionalities as services that can be consumed by the operating system, the boot loader or any other user provided application. And of course GRUB2 supported UEFI too.

Soon the Linux kernel gained the option of being compiled as an EFI binary, via a stub that can be attached to the kernel code. This implies that the kernel itself could be launched by the firmware directly, making the boot loader something optional in most of the cases.

Over time new and more straightforward boot loaders focused on UEFI appeared, like gummiboot. Later this code was integrated into systemd and renamed as systemd-boot.

The code is very simple. Many orders of magnitude simpler than GRUB2. It is basically a very small EFI binary that presents a menu with the different boot loader entries (text files described in the Boot Loader Specification or BLS for short), and a call to the UEFI LoadImage function to delegate the execution to the selected kernel.

This boot loader can also work with the new unified kernel images (UKI), that are files that aggregate in a single unit the kernel, the command line, and the initrd. Those UKIs can be very handy for image based distributions, and openSUSE plans to support them as well.

Providing systemd-boot as an alternative for GRUB2 is something that openSUSE wanted to do for a long time. In August 2023 there was an announcement on the Factory mailing list about Tumbleweed supporting systemd-boot.

The announcement references a wiki entry that explains how to migrate an installation using GRUB2 to systemd-boot manually. Soon after the announcement, yast-bootloader gained support for it for new installations.

Supporting another boot loader comes with a cost. As argued, the code base is smaller, with less bugs and more easy to reason about. But the UEFI dependency decreases the amount of supported architectures (x86-64 and aarch64). That problem can be very much alleviated by providing another patch for GRUB2 to support the BLS entries, so the architecture of the distribution after the boot loader can be independent of the boot loader itself. The good news is that the patch already exists, and could potentially be added into the package.

Another problem is that systemd-boot does not speak btrfs. As an EFI binary, it can read files only from a FAT32 file system. This limitation can be resolved by moving the kernel and the initrd into the EFI system partition (ESP).

Finally, there is also the consideration of supporting snapshots in Tumbleweed and transactions in MicroOS. From the boot loader the user should be able to select what snapshot to boot from, like it is actually possible to do when using GRUB2. Both concepts are implemented using btrfs subvolumes, and there is only a subset of kernel, command line, initrd combinations that are valid for each of those subvolumes.

For example, let’s say we have two snapshots in our system, and each of these represents a system that has two kernels installed. It is possible that those two kernels are not the same across all the snapshots. Maybe one of the upgrades replaced one kernel with a newer version. We need some tool that can do the bookkeeping required to associate the correct combination that will produce a successful boot into any of those snapshots, creating the boot entries under those restrictions.

This tool is sdbootutil. Every time snapper creates or destroys a snapshot (for example, when the system gets updated), it will call this tool that will analyze the content of the snapshots, making sure that the corresponding kernel is installed in the ESP, a valid initrd for this kernel is present (if not it will be created calling mkinitrd) and a boot entry is created that connects the kernel, the initrd and the snapshot via the command line. It also takes care of other details, like checking the free space on the partition.

Usually his process works transparently, but is good to remember that we can force a clean state with:

sdbootutil add-all-kernels
sdbootutil remove-all-kernels

Just in case, you know …

Full disk encryption

The other aspect that we want to announce is the support of full disk encryption (FDE) based on systemd.

FDE is not the new kid on the block. GRUB2 could unlock LUKS volumes since long ago using the cryptomount command. Traditionally this will request the password from the user two times: once when the boot loader does the unlock and again when the initrd does the same later. There are ways to avoid the second request injecting the password into the initrd or, if you are using the openSUSE package, it will inject the password transparently into the initrd.

Recently GRUB2 gained two new features: partial support of LUKS2 encrypted devices (using PBKDF2 as key derivation function instead of the more secure and recommended Argon2id) and a key protection mechanism that can store secrets in devices like the TPM2.

`TPM2`

Explaining how TPM2 works in detail is a topic for another post, but for now we can think of it as a crypto device that be used to unlock secrets only when certain conditions related to the state of the system are met. The TPM2 will unlock the secret if the system is in a healthy state.

This term is a technical one, and is related to assert that the system is in a known good state. In other words, we know for sure that the firmware has not been tampered with, the boot loader is the one that we installed and has not been replaced, that the kernel is exactly the one that comes from the distribution, that the kernel command line is the one that we expect, and that the initrd that we used does not contain any extra binary that we do not control.

Internally the TPM2 has some registers, known as platform configuration register (PCR). In the TPM2 specification there are 24 of them and the size of one is enough to store the value of a hash function, like SHA1 or SHA256. They are separated by banks: one per supported hash function, but this is too much detail for now.

Those registers are kind of special. We can reset them, usually setting the value to 0. We can read the value, or we can “extend” them. The write operation is designed in a way that we cannot set any random value in the register, except the result of the associated hash function concatenating the current PCR value and a new value provided by the user.

The current value of the PCR can only be produced by extending this register using exactly the same sequence of values. If we change even one bit of one of the values, we will produce a wildly different final result for the same PCR.

This feature is used in a process known as “measured boot”, where each stage in the boot chain is measured before it is executed. This means that before the initial stages of the firmware are running, there is a process that will calculate the hash of the code in memory, and extend one of the PCRs using this value. This is repeated until the very end of the boot sequence: the kernel and the initrd.

When measured boot is in place, the final values of the first 10 PCRs will contain values than can only be predicted if the machine is using a well known version of firmware, boot loader and kernel, together with the associated data like certificates, configuration files, or kernel parameters. If one of those elements change (for example, by using a different secure boot certificate), it will generate PCR values different from the ones that we expect.

TPM2 chips are very interesting devices, and the set of features go far beyond measured boot. If you want to learn more I recommend resources like this or this.

`TPM2` for `FDE`

Anyway, the gist here is that we can create a “policy” that can instruct the TPM2 to decrypt a secret only if certain PCRs contains the expected values. The details are a bit different, but for now lets use this model as a good first approximation.

The idea is that we can encrypt a password with the values of certain PCR registers, so GRUB2 can later attach the LUKS2 device if the TPM2 can recover the password, validating the health of the system until this point. If the TPM2 fails to decrypt it, that would mean that some PCR has not the expected value and some stage in the boot process changed. In this situation GRUB2 will ask the password from the user to continue loading the kernel and the rest of the system. It delegates the trust about the new state to the user.

GRUB2 also provides a tool to seal secrets under the current values of a subset of PCRs. This is nice but also presents several problems. One is that maybe we are setting the system up in a way that we know the PCRs values will change during the next boot (for example, during the first installation, a boot loader upgrade or a firmware update). In this case sealing the password using the current register values is useless: we need to be able to predict the new ones and use those hypothetical values to do the sealing.

The other problem is more insidious and will become critical later. The expected values can change frequently and can not be unique. Maybe there is a set of valid ones. We can choose to boot from a different kernel or from a different snapshot. The TPM2 provides a solution for this using something known as authorized policies. They are a way of creating policies that can change, but they are validated by a signature. In essence, we create a public and a private key, and we create multiple PCR policies that are signed using the private key. Now the TPM2 can validate the signature using the public part, and unseal the secret using the PCRs values stored in the new policy.

Since early 2023 openSUSE provides the pcr-oracle tool to help with the prediction of the PCR registers, and encrypt a key under those values using both PCR policies or authorized policies. Using this tool we can now seal a secret under a set of PCRs values that can change!

In the openSUSE wiki we can find more documentation about those topics, including instructions about how to use it in our installation.

Using `systemd` for disk encryption

With GRUB2 the FDE is working properly, so why look for something else? One reason is very evident: this architecture can only work … well … only if our openSUSE GRUB2 version is used. It will not work for other boot loaders like systemd-boot. In fact it will not work with the the upstream version of GRUB2 itself.

But there is a second reason: we can argue that there is not a full measured boot in place with GRUB2. If the boot loader needs to unlock the device before it can load the kernel, is natural that the PCR policies that will evaluate the health of the system cannot make asserts on the kernel, command line or initrd that will be used. Those will be loaded after the LUKS2 device has been opened.

The use of systemd-boot gives us an alternative architecture for FDE that can work properly with any boot loader that follows the BLS (remember, there is a patch for GRUB2 to support it somewhere, so it is not excluded a priori), and provides the chance to do a full measured boot attestation before unlocking the device.

One difference is that the kernel and the initrd will be placed in the unencrypted ESP, and the unlock of the sysroot will be done from inside the initrd using the different options that systemd-cryptsetup offers. Currently it can unlock the device using a normal password, a TPM2 with authorized policies (with optionally a PIN that must be entered by the user) or a FIDO2 key device. In the /etc/crypttab file we need to describe the unlocking mechanism.

pcr-oracle has been extended to support the creation of authorized policies that systemd can understand. They are stored in a JSON file that contains multiple predictions, each one of them indicating the PCRs involved, the TPM2 policy hash, the fingerprint of the public key and the signature of the policy. This, together with the public key PEM file, composes all the data required for systemd-cryptsetup to use the TPM2 for the unseal of the LUKS2 key.

The RSA 2048 key used to sign the policy can be created with openssl or with pcr-oracle itself. A note of caution: if the private key gets leaked, this is a game over for the expected security that the TPM2 could provide. Luckily the solution is cheap in this case: generate a new key, re-register the key in the LUKS2 key slot with systemd-cryptenroll and use sdbootutil to regenerate the predictions for each boot entry. Yeah … we will document all the process in the “systemd-fde” wiki page and provide better tools, but trust me, it is indeed a cheap operation.

openSUSE is providing a MicroOS image named kvm-and-xen-sdboot that shows how all of this is working. This image contains some of the already mentioned tools integrated and some other new ones:

systemd-boot: Boot loader used instead of the default GRUB2
sdbootutil: Helper scripts to synchronize the boot entries of the system
pcr-oracle: Predict the PCRs values for the next boot, and creates the authorized policies for systemd
disk-encryption-tool: Encrypt the device where sysroot is located on the first boot
dracut-pcr-signature: dracut module that will load the predictions into the initrd from the ESP

Those tools are designed to work together for this new FDE architecture. What follows is a brief description on how all is connected.

Once we get the new MicroOS qcow2 image and we setup the VM, we can proceed with the boot process. If the VM has a virtual TPM2 device it will start measuring the executed code and data, extending the corresponding PCRs. Once systemd-boot has been reached, it will find the correct boot entry for this session and will read the corresponding kernel and initrd from it.

At this moment the image is not encrypted. Inside the initrd that is used during this first boot, the disk-encryption-tool script will be called. Using some heuristics it will find the partition that belongs to sysroot (where the system is located), and will resize it to reserve 32MB for the LUKS2 header. After that it will use all the magic that cryptsetup provides to re-encrypt the device using a locally generated password. This password, as of today, corresponds to the recovery key that will be presented to the user at the end and the user should take note and keep it safe.

After the re-encryption, the system /etc/crypttab will be updated to communicate that this device is now encrypted and should be managed with different tools later.

At the end of the initrd we switch to the new sysroot, now finally located in an encrypted device. The disk-encryption-tool script already did its main job, but it installed two modules for jeos-firstboot, that will be executed on the first boot of the system, which is currently happening!

The first module, enroll, will detect if there is a FIDO2 key inserted and a TPM2 available. If so it will present a dialog asking what do you want to use to unlock the system. The second module will ask the user if the root password will also be enrolled in the LUKS2 header as a new key, and will show the recovery key generated earlier.

As of today it is not advisable to register both. As we described earlier the FIDO2 key will make more sense if we are using a laptop or a desktop machine and we want unlock the encrypted device with proof of a token that we own. This is an interactive process. The TPM2 makes more sense on situations where we do not want to interact with the system, and we want to automatically unlock the device only if we can assert the health of the system (no tamper occured in the boot chain).

If we register the FIDO2 key, systemd-cryptenroll will be called and we will be asked to press the button two times and the installation process will be over. At the next boot we will be required to present the key, and if the key is missing, the recovery password will be asked.

If we register the TPM2 device, a new RSA 2048 key gets generated and stored (the public and private parts) in /etc/systemd and systemd-cryptenroll will be used to enroll the public key and to annotate the PCRs that are used in the sealing of the LUKS2 key. By default we will be using 0, 2, 4, 7, and 9. You can check the meaning in this reference. PCRs 0 and 2 will measure all the UEFI firmware code. PCR 4 will measure the boot loader (systemd-boot) and the kernel (also UEFI binaries). PCR 7 will register all the secure boot certificates, and PCR 9 will be used by the kernel to measure the command line and the initrd.

This covers pretty much all that can make sense, but it is the user who has the final word on what to measure. The reason is that the predictions are done inside sdbootutil that, remember, will be automatically executed after each change in the system (updates, package removal, snapshots management, etc), and this tool will produce predictions only for the PCRs registered in the LUKS2 header.

Regardless of the selected unlocking mechanism, the /etc/crypttab file will be updated with this selection and a new initrd will be generated to contain this information for the next boot.

Finally, the last component, dracut-pcr-signature will be responsible that during the subsequent boots all the information that systemd-cryptsetup requires for the unlock will be present “on-the-fly” inside the initrd. It should be noted that the initrd will require the JSON file with the policies and the key, but those cannot be included in the initrd! The moment that we make a prediction of a PCR that is extended with the hash of the initrd, that is all, and we cannot touch the initrd anymore as this would produce a new hash and automatically will invalidate the prediction.

This dracut module will be executed before the systemd-cryptsetup generator for any encrypted devices has started, and will search in the ESP partition for a tpm2-pcr-signature.json file, that contains all the valid prediction for the current boot. Once this file is in place, the systemd-crypsetup will be able to assert the device in the current state is the expected one and the boot process can continue until the end.

Future

The image is here, and is a sound PoC. It provides a much more simple architecture and will place some components in the correct place. This will help a lot in the next stages, as there are some other things that we want to do with the distribution in relation to FDE.

One pretty clear disk-encryption-tool has limited use outside image based installation. Part of this code should be living in YaST and in Agama. The installer is already creating LUKS2 devices, so it should be “easy” to extend it in a way that works for us.

Ideally, the jeos-firstboot modules should also live in the installer, but somehow they make sense here too. In any case the functionality should not be separated, and both should be merged.

The encryption tool is doing something right from the very start: the master key, together with all the user keys are generated during installation time, but one possible improvement is generating the recovery key a bit later using the systemd tools. It is a small detail, but separating system keys from users keys can simplify the architecture.

Another aspect to improve is that the user may want to use the TPM2 and the FIDO2 key at the same time. For example, by default the TPM2 is used, and if the stage changed in a way that fails the prediction (or there is a security breach that has been detected), the user can delegate the unlock to the FIDO2 key, instead of using a password.

The sdbootutil script contains a bunch of features that should be also living in systemd. Working with upstream will make this tool obsolete with time, which would be more good news.

Another improvement that we can help with in systemd is to improve the diagnosis about the reasons making the TPM2 reject the unseal of the LUKS2 key. Today we have a general fail message without reporting what PCR or what measured component inside the PCR is reporting a different hash than the one predicted. This will help a lot understating what did go wrong. Was the boot loader changed? Or something in the firmware?

pcr-oracle is a very good tool for predicting the next PCR values. It was very easy to extend to parse the new events in the log related with the full measured boot process, including the kernel, systemd-boot extensions on PCR 12, or generating the JSON document required by systemd. The new systemd 255 (released a week ago from the time of writing this) includes a similar tool named systemd-pcrlock that can help us in providing the improved diagnosis that we are looking for. Evaluating this tool to do the predictions will be done soon too.

As today Type#1 and Type#2 entries from the BLS are not isomorphic. There are sections in the EFI file of the UKI format that do not exist in the text representation. Maybe we will decide to use UKIs in the future, or maybe not. So a good improvement is working on helping with this unification, that will (among other things) provide a standard way of splitting the JSON file and associating the predictions to each boot loader entry.

Generating and registering a new key, or selecting a different set of PCRs is today a manual process. The current tools can be extended to help in those processes, or better documentation could be provided.

The new approach for FDE is not about excluding GRUB2 from the equation. It is about providing a chance of using different boot loaders that follows the BLS. Validating that a proper patched (duh!) GRUB2 can work with all this is still something to be done.

Also, another thing that needs to be validated and improved are installations with multiple encrypted disks. In principle the design and the code is supporting it (even when the PCR registers per volume are different). openQA will do wonders here.

And finally, we should rethink if the UKIs do make sense for openSUSE or not. If we go in that direction, the private key used for signing the policies will be kept in OBS and those policies will also be generated in the build service, using a different set of PCR values.

In any case, there is a bunch of work ahead of us.

openSUSE MicroOS

Managing System Extensions with sysextmgrcli

Managing System Extensions on openSUSE MicroOS with sysextmgrcli

What is sysextmgrcli?

The Architecture: Smart Snapshots

Essential Commands

1. Listing and Checking Images

2. Installing New Extensions

3. Maintenance and Updates

The “Activation” Catch

Available default system extention (sysext) images:

Summary

You need git on your openSUSE MicroOS ?

You do not need ‘git’ anymore on your system ?

YaST2 support for Full Disk Encryption with TPM2

Intro

Hardware Requirement:

Installation of openSUSE MicroOS

Boot installation media

Finish FDE Setup

Re-enrollment

Next Steps

Further Documentation

zypper single transaction mode now default

Default zypper backend

How is this implemented?

What does this mean for the users?

Presenting GRUB2 BLS

Protecting against rogue devices in openSUSE with Full Disk Encryption

The fault in the design

Presenting GRUB2 BLS

GRUB2 with BLS is now in MicroOS and Tumbleweed

Quickstart in Full Disk Encryption with TPM and YaST2

Intro

Hardware Requirement:

Installation of openSUSE MicroOS

Boot installation media

Finish FDE Setup

Adjusting kernel boot parameters

Re-enrollment

Virtual Machines

Next Steps

Further Documentation

Configuration files in /etc and /usr/etc

Intro

RPM and Configuration Files

Atomic Updates

Image Based Updates

Goal

Requirements for a Solution

Solutions

Application Configuration Files

System Configuration Files (network, hardware, …)

System Databases (rpc, services, protocols)

/etc/passwd, /etc/group and /etc/shadow

Further Documentation

Introduction of soft-reboot

Intro

Current status

How to use

Installation and Configuration

How to enable

How to disable

Easily set up Passwordless SSH Authentication

Motivation

How does it look like?

Where can I get it?

What about other distros?

Systemd-boot and Full Disk Encryption with TPM and FIDO2

Systemd-boot and Full Disk Encryption in Tumbleweed and MicroOS

systemd-boot

Full disk encryption

TPM2

TPM2 for FDE

Using systemd for disk encryption

Future

Managing System Extensions on openSUSE MicroOS with `sysextmgrcli`

What is `sysextmgrcli`?

You need `git` on your openSUSE MicroOS ?

`TPM2`

`TPM2` for `FDE`

Using `systemd` for disk encryption