Quest to reduce firmware size

[tlaurion]

Hello guys/gals

There is tickets opened to:

• remove networking ( x230 x220 t430 t420 base boards: remove networking? #564 ), with corresponding PR attempt to remove networking with cleanup ( X230: remove networking (and cleanup) #572 ) while cryptsetup seemed to depend on CONFIG_NET and some explicit crypto modules in the past ( cryptsetup kernel backend requires CONFIG_NET #79 ), while putting CONFIG_LINUX_E1000E=n in board config liberates 295688 bytes. @merge : Is that enough?
• reduce libslang.so.2 footprint ( Reduce libslang.so.2 library footprint #517 )
• reduce kernel crypto footprint (what ticket?)

The priorities should be, see below for binaries/libraries footprints:

• Reduce libgcrypt.so.20 footprint (1145176 uncompressed bytes)
• Reduce lvm footprint (1599032 uncompressed bytes)
• Reduce gpg footprint (869568 uncompressed bytes)
• libcairo.so.2 footprint (851968 uncompressed bytes)
• libpixman-1.so.0 footprint (667048 uncompressed bytes)

EDIT: one liner to decompress and differenciate builds space

To help in finding consumed space, here is what I do.

make BOARD=x230
cd build/x230/
xz --decompress initrd.cpio.xz
cpio --extract < initrd.cpio
cpio --extract < heads.cpio
cpio --extract < modules.cpio

find . -type f -ls | sort -r -n -k7

789884 12288 -rw-r--r-- 1 user user 12582912 Jun 28 13:00 ./coreboot.rom
789885 11912 -rw-r--r-- 1 user user 12212224 Jun 28 13:00 ./initrd.cpio
789821 10648 -rw-r--r-- 1 user user 10903552 Jun 28 13:00 ./tools.cpio
789880 2868 -rw-r--r-- 1 user user 2936832 Jun 28 12:59 ./bzImage
789986 1564 -rwx------ 1 user user 1599032 Jun 28 13:01 ./bin/lvm
790106 1120 -rwx------ 1 user user 1145176 Jun 28 13:01 ./lib/libgcrypt.so.20
789881 968 -rw-r--r-- 1 user user 989696 Jun 28 12:59 ./modules.cpio
789943 852 -rwx------ 1 user user 869568 Jun 28 13:01 ./bin/gpg
790103 832 -rwx------ 1 user user 851968 Jun 28 13:01 ./lib/libcairo.so.2
790113 652 -rwx------ 1 user user 667048 Jun 28 13:01 ./lib/libpixman-1.so.0
790102 556 -rwx------ 1 user user 568680 Jun 28 13:01 ./lib/libc.so
789938 480 -rwx------ 1 user user 490640 Jun 28 13:01 ./bin/flashrom
789900 476 -rwx------ 1 user user 486024 Jun 28 13:01 ./bin/busybox
790018 384 -rwx------ 1 user user 391760 Jun 28 13:01 ./bin/scdaemon
789944 352 -rwx------ 1 user user 359960 Jun 28 13:01 ./bin/gpg-agent
790117 328 -rwx------ 1 user user 332504 Jun 28 13:01 ./lib/libtpm.so
790105 320 -rwx------ 1 user user 324784 Jun 28 13:01 ./lib/libdevmapper.so.1.02
789882 316 -rw-r--r-- 1 user user 321024 Jun 28 13:00 ./heads.cpio
790109 312 -rwx------ 1 user user 318464 Jun 28 13:01 ./lib/libmbedcrypto.so.0
790123 292 -rw------- 1 user user 295688 Jun 28 13:01 ./lib/modules/e1000e.ko
790108 216 -rwx------ 1 user user 217192 Jun 28 13:01 ./lib/libksba.so.8
790114 200 -rwx------ 1 user user 201096 Jun 28 13:01 ./lib/libpng16.so.16
789925 180 -rwx------ 1 user user 180960 Jun 28 13:01 ./bin/dropbear
790037 172 -rwx------ 1 user user 172544 Jun 28 13:01 ./bin/ssh
790127 168 -rw------- 1 user user 170512 Jun 28 13:01 ./lib/modules/mmc_core.ko
790131 160 -rw------- 1 user user 160792 Jun 28 13:01 ./lib/modules/xhci-hcd.ko
789960 156 -rwx------ 1 user user 158952 Jun 28 13:01 ./bin/kexec
790104 156 -rwx------ 1 user user 155744 Jun 28 13:01 ./lib/libcryptsetup.so.4
789924 132 -rwx------ 1 user user 133216 Jun 28 13:01 ./bin/dmsetup
790130 132 -rw------- 1 user user 132520 Jun 28 13:01 ./lib/modules/usb-storage.ko
790051 124 -rwx------ 1 user user 122992 Jun 28 13:01 ./bin/tpm
790107 120 -rwx------ 1 user user 120024 Jun 28 13:01 ./lib/libgpg-error.so.0
790097 108 -rw------- 1 user user 108800 Jun 28 13:01 ./etc/wordlist_en_eff.txt
790121 108 -rwx------ 1 user user 107512 Jun 28 13:01 ./lib/libz.so.1
790119 96 -rwx------ 1 user user 98064 Jun 28 13:01 ./lib/libusb-1.0.so.0
789983 76 -rwx------ 1 user user 73920 Jun 28 13:01 ./bin/lspci
790101 72 -rwx------ 1 user user 73416 Jun 28 13:01 ./lib/libassuan.so.0
790124 68 -rw------- 1 user user 66208 Jun 28 13:01 ./lib/modules/ehci-hcd.ko
789914 64 -rwx------ 1 user user 62680 Jun 28 13:01 ./bin/cryptsetup-reencrypt
789913 56 -rwx------ 1 user user 55760 Jun 28 13:01 ./bin/cryptsetup
790129 52 -rw------- 1 user user 53136 Jun 28 13:01 ./lib/modules/sdhci.ko
790004 52 -rwx------ 1 user user 52520 Jun 28 13:01 ./bin/pinentry-tty
790126 52 -rw------- 1 user user 49352 Jun 28 13:01 ./lib/modules/mmc_block.ko
789934 48 -rwx------ 1 user user 49032 Jun 28 13:01 ./bin/fbwhiptail
790112 48 -rwx------ 1 user user 48672 Jun 28 13:01 ./lib/libpci.so.3.5.4
790111 48 -rwx------ 1 user user 48672 Jun 28 13:01 ./lib/libpci.so.3
790115 48 -rwx------ 1 user user 45160 Jun 28 13:01 ./lib/libpopt.so.0
790116 44 -rwx------ 1 user user 43864 Jun 28 13:01 ./lib/libqrencode.so.3
790128 40 -rw------- 1 user user 38104 Jun 28 13:01 ./lib/modules/sdhci-pci.ko
790089 36 -rwx------ 1 user user 35654 Jun 28 13:01 ./etc/functions
790068 32 -rwx------ 1 user user 29584 Jun 28 13:01 ./bin/veritysetup
789977 32 -rwx------ 1 user user 29568 Jun 28 13:01 ./bin/libremkey_hotp_verification
789906 24 -rwx------ 1 user user 23696 Jun 28 13:01 ./bin/cbmem
789940 24 -rwx------ 1 user user 21064 Jun 28 13:01 ./bin/flashtool
790019 20 -rwx------ 1 user user 20400 Jun 28 13:01 ./bin/scp
790087 20 -rw------- 1 user user 19992 Jun 28 13:01 ./etc/distro/keys/tails.key
790118 20 -rwx------ 1 user user 18800 Jun 28 13:01 ./lib/libusb-0.1.so.4
789949 20 -rwx------ 1 user user 16936 Jun 28 13:01 ./bin/gui-init
789904 16 -rwx------ 1 user user 15368 Jun 28 13:01 ./bin/cbfs
790110 16 -rwx------ 1 user user 14912 Jun 28 13:01 ./lib/libnpth.so.0
790056 16 -rwx------ 1 user user 14464 Jun 28 13:01 ./bin/uefi
790120 16 -rwx------ 1 user user 13984 Jun 28 13:01 ./lib/libuuid.so.1
790085 16 -rw------- 1 user user 12413 Jun 28 13:01 ./etc/distro/keys/fedora.key
789879 12 -rw-r--r-- 1 user user 11225 Jun 28 13:00 ./hashes.txt
790007 12 -rwx------ 1 user user 10776 Jun 28 13:01 ./bin/poke
790125 12 -rw------- 1 user user 10592 Jun 28 13:01 ./lib/modules/ehci-pci.ko
790132 12 -rw------- 1 user user 10456 Jun 28 13:01 ./lib/modules/xhci-pci.ko
789931 12 -rwx------ 1 user user 10409 Jun 28 13:01 ./bin/factory-reset-libremkey.sh
789969 12 -rwx------ 1 user user 10093 Jun 28 13:01 ./bin/kexec-select-boot
789945 12 -rwx------ 1 user user 8780 Jun 28 13:01 ./bin/gpg-gui.sh
790014 12 -rwx------ 1 user user 8581 Jun 28 13:01 ./bin/reencrypt-luks
789966 8 -rwx------ 1 user user 7944 Jun 28 13:01 ./bin/kexec-save-default
790002 8 -rwx------ 1 user user 7872 Jun 28 13:01 ./bin/peek
789993 8 -rwx------ 1 user user 7556 Jun 28 13:01 ./bin/mount-sdcard
790049 8 -rwx------ 1 user user 7448 Jun 28 13:01 ./bin/totp
789957 8 -rwx------ 1 user user 7064 Jun 28 13:01 ./bin/hotp
789896 8 -rwx------ 1 user user 6832 Jun 28 13:01 ./bin/base32
790011 8 -rwx------ 1 user user 5304 Jun 28 13:01 ./bin/qrenc
789968 8 -rwx------ 1 user user 4972 Jun 28 13:01 ./bin/kexec-seal-key
789994 8 -rwx------ 1 user user 4185 Jun 28 13:01 ./bin/mount-usb
789965 4 -rwx------ 1 user user 3654 Jun 28 13:01 ./bin/kexec-parse-boot
790098 4 -rwx------ 1 user user 3622 Jun 28 13:01 ./init
789910 4 -rwx------ 1 user user 3329 Jun 28 13:01 ./bin/config-gui.sh
789961 4 -rwx------ 1 user user 3099 Jun 28 13:01 ./bin/kexec-boot
790020 4 -rwx------ 1 user user 2753 Jun 28 13:01 ./bin/seal-libremkey
789970 4 -rwx------ 1 user user 2628 Jun 28 13:01 ./bin/kexec-sign-config
789962 4 -rwx------ 1 user user 2379 Jun 28 13:01 ./bin/kexec-insert-key
789937 4 -rwx------ 1 user user 2107 Jun 28 13:01 ./bin/flash.sh
789936 4 -rwx------ 1 user user 2077 Jun 28 13:01 ./bin/flash-gui.sh
790066 4 -rwx------ 1 user user 2070 Jun 28 13:01 ./bin/usb-scan
789964 4 -rwx------ 1 user user 2033 Jun 28 13:01 ./bin/kexec-parse-bls
790021 4 -rwx------ 1 user user 2027 Jun 28 13:01 ./bin/seal-totp
790061 4 -rwx------ 1 user user 1839 Jun 28 13:01 ./bin/unseal-hotp
789883 4 -rw------- 1 user user 1748 Jun 28 13:01 ./.ash_history
789967 4 -rwx------ 1 user user 1689 Jun 28 13:01 ./bin/kexec-save-key
790086 4 -rw------- 1 user user 1629 Jun 28 13:01 ./etc/distro/keys/qubes-4.key
789921 4 -rwx------ 1 user user 1552 Jun 28 13:01 ./bin/diceware.sh
789963 4 -rwx------ 1 user user 1375 Jun 28 13:01 ./bin/kexec-iso-init
790134 4 -rwx------ 1 user user 1373 Jun 28 13:01 ./mount-boot
789941 4 -rwx------ 1 user user 1299 Jun 28 13:01 ./bin/generic-init
789971 4 -rwx------ 1 user user 1119 Jun 28 13:01 ./bin/kexec-unseal-key
790151 4 -rwx------ 1 user user 1000 Jun 28 13:01 ./sbin/insmod
790052 4 -rwx------ 1 user user 983 Jun 28 13:01 ./bin/tpm-reset
789976 4 -rwx------ 1 user user 965 Jun 28 13:01 ./bin/libremkey_hotp_initialize
790140 4 -rwx------ 1 user user 922 Jun 28 13:01 ./sbin/config-dhcp.sh
789972 4 -rwx------ 1 user user 875 Jun 28 13:01 ./bin/key-init
789905 4 -rwx------ 1 user user 799 Jun 28 13:01 ./bin/cbfs-init
790082 4 -rw------- 1 user user 736 Jun 28 13:01 ./etc/config
789997 4 -rwx------ 1 user user 675 Jun 28 13:01 ./bin/network-init-recovery
790057 4 -rwx------ 1 user user 661 Jun 28 13:01 ./bin/uefi-init
790062 4 -rwx------ 1 user user 634 Jun 28 13:01 ./bin/unseal-totp
790074 4 -rwx------ 1 user user 574 Jun 28 13:01 ./bin/x230-flash.init
790012 4 -rwx------ 1 user user 366 Jun 28 13:01 ./bin/qubes-measure-luks
789939 4 -rwx------ 1 user user 360 Jun 28 13:01 ./bin/flashrom-kgpe-d16-openbmc.sh
790092 4 -rwx------ 1 user user 340 Jun 28 13:01 ./etc/keylime-init
790072 4 -rwx------ 1 user user 320 Jun 28 13:01 ./bin/wget-measure.sh
790013 4 -rwx------ 1 user user 258 Jun 28 13:01 ./bin/reboot
790065 4 -rwx------ 1 user user 220 Jun 28 13:01 ./bin/usb-init
790008 4 -rwx------ 1 user user 205 Jun 28 13:01 ./bin/poweroff
790088 4 -rw------- 1 user user 197 Jun 28 13:01 ./etc/fstab
790093 4 -rw------- 1 user user 174 Jun 28 13:01 ./etc/motd
789946 4 -rwx------ 1 user user 106 Jun 28 13:01 ./bin/gpgv
789888 4 -rw------- 1 user user 73 Jun 28 13:01 ./.gnupg/gpg-agent.conf
789920 4 -rwx------ 1 user user 62 Jun 28 13:01 ./bin/diceware-eff.sh
790073 4 -rwx------ 1 user user 35 Jun 28 13:01 ./bin/whiptail
790096 4 -rw------- 1 user user 27 Jun 28 13:01 ./etc/shells
790095 4 -rw------- 1 user user 27 Jun 28 13:01 ./etc/passwd
790091 4 -rw------- 1 user user 20 Jun 28 13:01 ./etc/hosts
790090 4 -rw------- 1 user user 10 Jun 28 13:01 ./etc/group
789889 4 -rw------- 1 user user 10 Jun 28 13:01 ./.gnupg/gpg.conf

[snmcmillan]

A few other ideas:

1. Clean up board-specific files from other devices. (Example is flashrom-kgpe-d16-openbmc.sh is present in X230 builds, x230-flash.init being in main X230 or on Chell Chromebook). This change will likely need to happen during the build process.

2. Move flash definitions from flash.sh to the board config files. As we support more boards, this will be critical, as all maintained boards are presently defined in flash.sh, and as this project supports more and more boards, flash.sh will get larger and larger. I've started work on this.

These may not be as high priority as other changes, but ones to certainly look at to help reduce firmware size, especially as the project grows.

[tlaurion]

@SebastianMcMillan : Those are good cleanup ideas, but unfortunately, will not impact the initrd.cpio.xz (including heads.cpio and tools.cpio), those text files being highly compressed.

The problem lies in binaries, not higly compressed under initrd.cpio.xz

[owlshrimp]

This article (or the series it comes from) may yield some inspiration:
https://lwn.net/Articles/748198/
Edit: this one is probably more useful https://lwn.net/Articles/741494/

[tlaurion]

Solution lies in #307

[tlaurion]

@zaolin pointed out:

Update: Getting rid of libgcrypt and replacement for gpg would be a good way to safe 2MB in total.
See, https://sequoia-pgp.org/ as alternative