Build all packages with cproc

[michaelforney]

TODO:

• alsa-lib
  Needs VLAs. Port things to tinyalsa instead?
• catgirl
  Needs wide string literals.
• efivar
• f2fs-tools
• mpv
  Needs atomic types.
• musl
• nginx
• pixman
• python
• the_silver_searcher
  Needs TLS. Could be ported to pthread_key.

Done:

• acme-client
• alsa-utils
• atinout
• awk
• bc
• bearssl
• binutils
• blind
• byacc
• bzip2
• cmark
• cproc
• curl
• dmenu
• dnssec-rr
• dosfstools
• e2fsprogs
• efibootmgr
• elftoolchain
• expat
• farbfeld
• ffmpeg
• file
• flex
• fontconfig
• freetype
• fribidi
• fuse
• git
• hostap
• ii
• iproute2
• jbig2dec
• kbd
• less
• libass
• libcbor
• libdrm
• libevdev
• libevent
• libffi
• libfido2
• libgpiod
• libinput
• libjpeg-turbo
• libliftoff
• libnl
• libpciaccess
• libpng
• libressl
• libtermkey
• libtls-bearssl
• libutp
• libxkbcommon
• lpeg
• lua
• make
• mandoc
• mc
• monocypher
• msmtp
• mtdev
• mupdf
• nasm
• ncompress
• ncurses
• netsurf
• nsd
• oksh
• openbsd
• openntpd
• openssh
• pcre
• perp
• pigz
• pounce
• qbe
• rc
• samurai
• sbase
• sdhcp
• sinit
• sshfs
• squashfs-tools-ng
• st
• strace
• swc
• syslinux
• texi2mdoc
• tinyemu
• transmission
• tz
• u-boot
• ubase
• unzip
• utf8proc
• util-linux
• velox
• vis
• wayland
• wld
• xz
• zlib
• zstd

N/A:

• adobe-source-fonts
• cacert
• linux-headers
• man-pages-posix
• skeleton
• terminus-font
• usbids
• wayland-protocols
• xkeyboard-config
• youtube-dl

Hi! I'm not an oasis user, but I'm curious - does Linux compile under cproc, or is it still clang or gcc exclusive?

[michaelforney]

No, Linux only builds with gcc or clang. Maybe someday in the far far future, cproc would be capable enough to build it, but for now the goal is just to build the userspace packages.

[firasuke]

Why an additional compiler? Also, how does it optimize for both size and performance compared to gcc and clang?

[michaelforney]

Why an additional compiler?

I don't quite understand your question. gcc and clang can and will continue to be able to build all of oasis, no additional compiler is needed.

However, if you are asking why I'd want to be able to build the oasis userspace with a compiler written in 7000 lines of C as opposed to one written in 5 million lines of C++, then I don't really know how to answer.

Of course I'd like to be able to build Linux with cproc. It just doesn't seem like a realistic goal at this point.

Also, how does it optimize for both size and performance compared to gcc and clang?

I haven't done much testing of this, but QBE's goal is "70% of the performance of advanced compilers in 10% of the code". So far, I've just been trying to get things to build and work (regardless of size and performance).

[firasuke]

I see, thanks for clarifying.

I agree that having a much lighter compiler is beneficial (compilation-time wise), but aren't binaries (and the degree of optimization) what matter here, and not the compiler itself?

[michaelforney]

The main goal of oasis is not to build the most optimized Linux system possible. It is to provide a complete system built from components that have small and approachable codebases. The compiler is just one of those components.

That said, good performance is still a goal, and most of the time this falls out of using smaller implementations of software. This allows you to use a simpler compiler without sacrificing too much.

The compiler you use can be configured in config.lua. You'll still be able to build gcc with whatever fancy CFLAGS you want. This issue is just about making cproc an usable option as well.

[ddevault]

Maybe someday in the far far future, cproc would be capable enough to build it

Correction: maybe someday Linux will be compliant enough to be built with a C compiler 😉

[nwmcsween]

Do you have any comparisons with resulting size when using cproc (wondering where it stands wrt gcc)?

[michaelforney]

Here's a size comparison of the core packages with gcc -Os (left) and cproc (right):

 bin/awk                                | Bin 204104 -> 239496 bytes
 bin/basename                           | Bin 34240 -> 33952 bytes
 bin/bc                                 | Bin 123960 -> 159464 bytes
 bin/blkdiscard                         | Bin 25792 -> 25504 bytes
 bin/bridge                             | Bin 137016 -> 199576 bytes
 bin/bzip2                              | Bin 99104 -> 169568 bytes
 bin/cal                                | Bin 50904 -> 50616 bytes
 bin/cat                                | Bin 25792 -> 25504 bytes
 bin/catargs                            | Bin 13240 -> 8784 bytes
 bin/catenv                             | Bin 13304 -> 13008 bytes
 bin/catlimits                          | Bin 17472 -> 14256 bytes
 bin/catuid                             | Bin 17336 -> 13008 bytes
 bin/chgrp                              | Bin 50464 -> 54432 bytes
 bin/chmod                              | Bin 46272 -> 50304 bytes
 bin/chown                              | Bin 54560 -> 58624 bytes
 bin/chroot                             | Bin 25792 -> 25536 bytes
 bin/chvt                               | Bin 29984 -> 29728 bytes
 bin/cksum                              | Bin 38336 -> 36032 bytes
 bin/clear                              | Bin 26048 -> 25728 bytes
 bin/cmp                                | Bin 34496 -> 38400 bytes
 bin/cols                               | Bin 46880 -> 46720 bytes
 bin/comm                               | Bin 38592 -> 38336 bytes
 bin/cp                                 | Bin 42432 -> 42560 bytes
 bin/cron                               | Bin 59064 -> 59032 bytes
 bin/ctrlaltdel                         | Bin 25792 -> 25472 bytes
 bin/curl                               | Bin 669960 -> 1223576 bytes
 bin/cut                                | Bin 46784 -> 47040 bytes
 bin/date                               | Bin 54840 -> 54616 bytes
 bin/dd                                 | Bin 46368 -> 42272 bytes
 bin/deallocvt                          | Bin 38464 -> 34496 bytes
 bin/demandoc                           | Bin 274520 -> 377048 bytes
 bin/df                                 | Bin 58816 -> 58752 bytes
 bin/diff                               | Bin 121912 -> 131448 bytes
 bin/dirname                            | Bin 34240 -> 33952 bytes
 bin/dmesg                              | Bin 42432 -> 38176 bytes
 bin/doas                               | Bin 116312 -> 122712 bytes
 bin/du                                 | Bin 46624 -> 50688 bytes
 bin/dumpkeys                           | Bin 124832 -> 136352 bytes
 bin/e2fsck                             | Bin 558128 -> 989544 bytes
 bin/echo                               | Bin 34240 -> 33920 bytes
 bin/ed                                 | Bin 100040 -> 108520 bytes
 bin/eject                              | Bin 25792 -> 25504 bytes
 bin/env                                | Bin 38336 -> 38112 bytes
 bin/expand                             | Bin 46880 -> 46848 bytes
 bin/expr                               | Bin 91680 -> 104128 bytes
 bin/fallocate                          | Bin 29984 -> 29664 bytes
 bin/false                              | Bin 13176 -> 8728 bytes
 bin/fdisk                              | Bin 422040 -> 811928 bytes
 bin/fgconsole                          | Bin 38504 -> 34472 bytes
 bin/find                               | Bin 84616 -> 85256 bytes
 bin/flock                              | Bin 29888 -> 29632 bytes
 bin/fmt                                | Bin 55008 -> 59168 bytes
 bin/fold                               | Bin 46880 -> 42560 bytes
 bin/free                               | Bin 26048 -> 25888 bytes
 bin/freeramdisk                        | Bin 25792 -> 25472 bytes
 bin/fsfreeze                           | Bin 25792 -> 25504 bytes
 bin/getconf                            | Bin 46528 -> 45024 bytes
 bin/getkeycodes                        | Bin 34240 -> 30400 bytes
 bin/getty                              | Bin 42432 -> 42400 bytes
 bin/git                                | Bin 2645712 -> 7684128 bytes
 bin/grep                               | Bin 91840 -> 100064 bytes
 bin/halt                               | Bin 25792 -> 25472 bytes
 bin/head                               | Bin 46880 -> 42496 bytes
 bin/hostname                           | Bin 34240 -> 33952 bytes
 bin/hotplugd                           | Bin 29888 -> 29760 bytes
 bin/hwclock                            | Bin 50648 -> 50456 bytes
 bin/id                                 | Bin 50624 -> 50528 bytes
 bin/insmod                             | Bin 38080 -> 37888 bytes
 bin/install                            | Bin 54560 -> 54496 bytes
 bin/ip                                 | Bin 533656 -> 1277976 bytes
 bin/join                               | Bin 50976 -> 51136 bytes
 bin/kbd_mode                           | Bin 34240 -> 34592 bytes
 bin/kbdinfo                            | Bin 38496 -> 35008 bytes
 bin/kill                               | Bin 42880 -> 38656 bytes
 bin/killall5                           | Bin 58912 -> 59072 bytes
 bin/ksh                                | Bin 243536 -> 355504 bytes
 bin/last                               | Bin 46544 -> 42192 bytes
 bin/lastlog                            | Bin 50640 -> 50448 bytes
 bin/link                               | Bin 25792 -> 25472 bytes
 bin/ln                                 | Bin 29888 -> 29760 bytes
 bin/loadkeys                           | Bin 170464 -> 205184 bytes
 bin/logger                             | Bin 59480 -> 58968 bytes
 bin/login                              | Bin 87240 -> 87208 bytes
 bin/logname                            | Bin 34240 -> 33952 bytes
 bin/losetup                            | Bin 202312 -> 378856 bytes
 bin/ls                                 | Bin 67064 -> 75448 bytes
 bin/lsmod                              | Bin 38336 -> 38112 bytes
 bin/lsusb                              | Bin 54720 -> 58720 bytes
 bin/lua5.2                             | Bin 231128 -> 322232 bytes
 bin/m4                                 | Bin 137928 -> 147336 bytes
 bin/mandoc                             | Bin 524696 -> 706776 bytes
 bin/md5sum                             | Bin 46848 -> 46976 bytes
 bin/mesg                               | Bin 26048 -> 29856 bytes
 bin/mkdir                              | Bin 29888 -> 33728 bytes
 bin/mke2fs                             | Bin 435600 -> 751048 bytes
 bin/mkfifo                             | Bin 29888 -> 29600 bytes
 bin/mknod                              | Bin 34080 -> 29696 bytes
 bin/mkswap                             | Bin 42432 -> 38272 bytes
 bin/mktemp                             | Bin 42440 -> 38248 bytes
 bin/mount                              | Bin 63328 -> 67488 bytes
 bin/mountpoint                         | Bin 58816 -> 58624 bytes
 bin/mv                                 | Bin 46528 -> 54976 bytes
 bin/nc                                 | Bin 268216 -> 658616 bytes
 bin/nice                               | Bin 29984 -> 29664 bytes
 bin/nl                                 | Bin 96096 -> 104672 bytes
 bin/nohup                              | Bin 29888 -> 29632 bytes
 bin/nologin                            | Bin 17600 -> 25472 bytes
 bin/ntpd                               | Bin 313048 -> 694584 bytes
 bin/od                                 | Bin 46848 -> 46816 bytes
 bin/openvt                             | Bin 59208 -> 64232 bytes
 bin/pagesize                           | Bin 26048 -> 25728 bytes
 bin/passwd                             | Bin 83400 -> 87880 bytes
 bin/paste                              | Bin 46784 -> 46912 bytes
 bin/patch                              | Bin 93032 -> 106440 bytes
 bin/pathchk                            | Bin 42176 -> 38112 bytes
 bin/pax                                | Bin 195512 -> 244024 bytes
 bin/perpboot                           | Bin 37896 -> 40456 bytes
 bin/perpctl                            | Bin 29696 -> 32400 bytes
 bin/perpd                              | Bin 70664 -> 81624 bytes
 bin/perphup                            | Bin 21504 -> 23608 bytes
 bin/perpls                             | Bin 42440 -> 49480 bytes
 bin/perpok                             | Bin 29704 -> 32304 bytes
 bin/perpstat                           | Bin 29768 -> 32224 bytes
 bin/pidof                              | Bin 58816 -> 58880 bytes
 bin/pigz                               | Bin 182112 -> 249664 bytes
 bin/pivot_root                         | Bin 25792 -> 25472 bytes
 bin/printenv                           | Bin 34240 -> 33952 bytes
 bin/printf                             | Bin 54816 -> 55136 bytes
 bin/ps                                 | Bin 75224 -> 75544 bytes
 bin/psfxtable                          | Bin 50880 -> 55968 bytes
 bin/pwd                                | Bin 34240 -> 33952 bytes
 bin/pwdx                               | Bin 26048 -> 25728 bytes
 bin/rc                                 | Bin 76192 -> 97288 bytes
 bin/readahead                          | Bin 33984 -> 33664 bytes
 bin/readlink                           | Bin 38336 -> 38080 bytes
 bin/renice                             | Bin 46368 -> 46080 bytes
 bin/resize2fs                          | Bin 322512 -> 562280 bytes
 bin/respawn                            | Bin 29888 -> 33760 bytes
 bin/rev                                | Bin 38592 -> 38336 bytes
 bin/rm                                 | Bin 42176 -> 46176 bytes
 bin/rmdir                              | Bin 25792 -> 25472 bytes
 bin/rmmod                              | Bin 25792 -> 25504 bytes
 bin/rsync                              | Bin 165064 -> 221512 bytes
 bin/runargs                            | Bin 33792 -> 36024 bytes
 bin/runargv0                           | Bin 29696 -> 31832 bytes
 bin/runchoom                           | Bin 33792 -> 36344 bytes
 bin/rundetach                          | Bin 29696 -> 31864 bytes
 bin/rundeux                            | Bin 37896 -> 36224 bytes
 bin/runenv                             | Bin 46080 -> 48656 bytes
 bin/runfile                            | Bin 33792 -> 36056 bytes
 bin/runlimit                           | Bin 42504 -> 42592 bytes
 bin/runlock                            | Bin 33792 -> 36104 bytes
 bin/runpause                           | Bin 29696 -> 31896 bytes
 bin/runsession                         | Bin 29696 -> 31832 bytes
 bin/runtool                            | Bin 62472 -> 72024 bytes
 bin/runtrap                            | Bin 37896 -> 37840 bytes
 bin/runuid                             | Bin 46088 -> 48760 bytes
 bin/samu                               | Bin 95784 -> 103752 bytes
 bin/scp                                | Bin 161368 -> 218200 bytes
 bin/sdhcp                              | Bin 50664 -> 46728 bytes
 bin/sed                                | Bin 112832 -> 125696 bytes
 bin/seq                                | Bin 50720 -> 46432 bytes
 bin/setfont                            | Bin 91680 -> 105152 bytes
 bin/setkeycodes                        | Bin 38336 -> 34464 bytes
 bin/setsid                             | Bin 25792 -> 25504 bytes
 bin/setvtrgb                           | Bin 59264 -> 59744 bytes
 bin/sftp                               | Bin 190664 -> 280472 bytes
 bin/sha1sum                            | Bin 46848 -> 46720 bytes
 bin/sha224sum                          | Bin 46848 -> 42880 bytes
 bin/sha256sum                          | Bin 46848 -> 42880 bytes
 bin/sha384sum                          | Bin 46848 -> 47360 bytes
 bin/sha512-224sum                      | Bin 46848 -> 47360 bytes
 bin/sha512-256sum                      | Bin 46848 -> 47360 bytes
 bin/sha512sum                          | Bin 46848 -> 43264 bytes
 bin/showconsolefont                    | Bin 46528 -> 47648 bytes
 bin/showkey                            | Bin 38560 -> 39104 bytes
 bin/sinit                              | Bin 17600 -> 17408 bytes
 bin/sleep                              | Bin 29984 -> 29600 bytes
 bin/sort                               | Bin 67136 -> 67584 bytes
 bin/split                              | Bin 42784 -> 42784 bytes
 bin/sponge                             | Bin 29896 -> 29640 bytes
 bin/ss                                 | Bin 221528 -> 310328 bytes
 bin/ssh                                | Bin 776776 -> 1451608 bytes
 bin/ssh-add                            | Bin 431848 -> 870072 bytes
 bin/ssh-agent                          | Bin 403176 -> 824088 bytes
 bin/ssh-keygen                         | Bin 493704 -> 982776 bytes
 bin/ssh-keyscan                        | Bin 542920 -> 1051160 bytes
 bin/sshd                               | Bin 858888 -> 1619032 bytes
 bin/stat                               | Bin 46544 -> 46512 bytes
 bin/stty                               | Bin 42528 -> 44672 bytes
 bin/swaplabel                          | Bin 42432 -> 38336 bytes
 bin/swapoff                            | Bin 54464 -> 54176 bytes
 bin/swapon                             | Bin 54464 -> 54208 bytes
 bin/switch_root                        | Bin 38080 -> 38048 bytes
 bin/sync                               | Bin 25792 -> 25440 bytes
 bin/sysctl                             | Bin 42432 -> 42304 bytes
 bin/syslogd                            | Bin 47136 -> 42304 bytes
 bin/tail                               | Bin 46624 -> 46592 bytes
 bin/tee                                | Bin 42176 -> 37920 bytes
 bin/test                               | Bin 38752 -> 34400 bytes
 bin/tftp                               | Bin 62664 -> 62760 bytes
 bin/time                               | Bin 33984 -> 29664 bytes
 bin/tinylog                            | Bin 54424 -> 59504 bytes
 bin/touch                              | Bin 46424 -> 46328 bytes
 bin/tr                                 | Bin 56832 -> 57152 bytes
 bin/true                               | Bin 13176 -> 8728 bytes
 bin/truncate                           | Bin 29888 -> 29696 bytes
 bin/tsort                              | Bin 42688 -> 42592 bytes
 bin/tty                                | Bin 34240 -> 33952 bytes
 bin/umount                             | Bin 58560 -> 58400 bytes
 bin/uname                              | Bin 34240 -> 33952 bytes
 bin/unexpand                           | Bin 46880 -> 46848 bytes
 bin/uniq                               | Bin 46880 -> 42688 bytes
 bin/unlink                             | Bin 25792 -> 25472 bytes
 bin/unshare                            | Bin 25792 -> 25504 bytes
 bin/uptime                             | Bin 42456 -> 42264 bytes
 bin/uudecode                           | Bin 46784 -> 47392 bytes
 bin/uuencode                           | Bin 38592 -> 38400 bytes
 bin/vmstat                             | Bin 62936 -> 67544 bytes
 bin/vtallow                            | Bin 25792 -> 25504 bytes
 bin/watch                              | Bin 38336 -> 38144 bytes
 bin/wc                                 | Bin 42784 -> 38464 bytes
 bin/which                              | Bin 42432 -> 38176 bytes
 bin/who                                | Bin 50640 -> 54480 bytes
 bin/whoami                             | Bin 42432 -> 42176 bytes
 bin/xargs                              | Bin 46880 -> 50976 bytes
 bin/xz                                 | Bin 215016 -> 452328 bytes
 bin/yacc                               | Bin 112776 -> 135016 bytes
 bin/yes                                | Bin 26048 -> 25728 bytes
 bin/zic                                | Bin 91872 -> 113408 bytes
 libexec/applyperms                     | Bin 50624 -> 50720 bytes
 libexec/git-core/git-bugreport         | Bin 1674928 -> 4363424 bytes
 libexec/git-core/git-credential-store  | Bin 1646224 -> 4308064 bytes
 libexec/git-core/git-daemon            | Bin 1666896 -> 4331648 bytes
 libexec/git-core/git-fast-import       | Bin 1688016 -> 4375712 bytes
 libexec/git-core/git-http-backend      | Bin 1654864 -> 4335136 bytes
 libexec/git-core/git-imap-send         | Bin 2114288 -> 5279360 bytes
 libexec/git-core/git-remote-http       | Bin 2126224 -> 5321344 bytes
 libexec/git-core/git-sh-i18n--envsubst | Bin 1646224 -> 4303712 bytes
 libexec/git-core/git-shell             | Bin 1650448 -> 4308480 bytes
 libexec/hotplugd-trigger               | Bin 38112 -> 37792 bytes
 libexec/mergeperms                     | Bin 42176 -> 41952 bytes
 libexec/sftp-server                    | Bin 165560 -> 234040 bytes
 libexec/shutdown                       | Bin 54464 -> 54208 bytes
 libexec/ssh-sk-helper                  | Bin 505584 -> 995608 bytes
 245 files changed, 0 insertions(+), 0 deletions(-)

This was with a local patch to qbe to put zero data in .bss instead of .data. For the most part it is pretty comparable, but there are some binaries that are quite a bit bigger when built with cproc. It'd be pretty interesting to investigate why that is.

[jpivarski]

The mean gcc size is 7.5% smaller, though the per-program variation is large enough that 27% smaller and 12% bigger are within one standard deviation in each direction. I'd call that "pretty comparable," too.

[173duprot]

I'm still very excited too see where this goes if your still working on it