busybox in bootstrap-tools doesn't work in WSL

[hsenag]

Issue description

Windows Subsystem for Linux doesn't currently support 32-bit binaries:
microsoft/WSL#390

Since x86_64 bootstrap-tools includes the i686 busybox, any build that requires it fails:

https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/linux/bootstrap-files/x86_64.nix#L2

Could it be changed to use an x86_64 specific busybox? With that hacked locally I got a lot further using WSL with nix. I'm happy to produce a patch myself but I'm not sure what the official way to update bootstrap-files is.

Steps to reproduce

Do something that requires bootstrap-tools.

Technical details

• System: (NixOS: nixos-version, Ubuntu/Fedora: lsb_release -a, ...)
  Ubuntu running under WSL (Windows 10 Creators Update):

ganesh@Priory:/mnt/c/msys64/home/ganesh/nixpkgs/general$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.5 LTS
Release:        14.04
Codename:       trusty

• Nix version: (run nix-env --version)
  nix-env (Nix) 1.11.8
• Nixpkgs version: (run nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion)
  "17.09pre105126.efe12d6658"

[andyli]

Same issue here. I'm also running WSL with Windows 10 Creators Update, but I've a clean install of Xenial instead of Trusty:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:        16.04
Codename:       xenial
$ nix-env --version
nix-env (Nix) 1.11.9
$ nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion
"17.09pre106045.7369fd0b51"

[pikajude]

@hsenag can you post what you have working at the moment? I've also run into this issue

[hsenag]

I dropped the import i686.nix and replaced it with this:

   busybox = import <nix/fetchurl.nix> {
      # made by just building busybox in an existing x86_64 nix install
     url = http://urchin.earth.li/~ganesh/temp/busybox;
     sha256 = "1wdhy09h4csm7vjk05s5sc8a9wypbv94ln4ikzjvyayrminky5sg";
     executable = true;
  };

You're welcome to refer to my URL if you want (no promises about how long it'll stay) or you can do something similar.

[puffnfresh]

I want to reference the x86_64 version of busybox in here:

nixpkgs/pkgs/stdenv/linux/bootstrap-files/x86_64.nix

Line 6 in 042ca21

url = http://tarballs.nixos.org/stdenv-linux/x86_64/4907fc9e8d0d82b28b3c56e3a478a2882f1d700f/bootstrap-tools.tar.xz;

But http://tarballs.nixos.org/stdenv-linux/x86_64/4907fc9e8d0d82b28b3c56e3a478a2882f1d700f/busybox is giving 404 - is that where it should be?

[dtzWill]

Partially to avoid this issue, musl bootstrap uses a 64bit busybox. Here's the latest info:

https://github.com/NixOS/nixpkgs/pull/36384/files#diff-e5cfbf7cb5224d730a5ce732c0850a0dR3

If you try it, please let us/me know if it works for you!

[eqyiel]

@dtzWill thank you, I had the same problem as @hsenag and using busybox from x86_64-musl.nix worked for me.

Changes in 0e49a63^..0e49a63
1 file changed, 3 insertions(+), 2 deletions(-)
pkgs/stdenv/linux/bootstrap-files/x86_64.nix | 5 +++--

modified   pkgs/stdenv/linux/bootstrap-files/x86_64.nix
@@ -1,5 +1,6 @@
-# Use busybox for i686-linux since it works on x86_64-linux as well.
-(import ./i686.nix) //
+# i686 doesn't work on WSL right now.  Use the x86_64 busybox from x86_64-musl instead.
+# See https://github.com/NixOS/nixpkgs/issues/24954#issuecomment-370871154
+(import ./x86_64-musl.nix) //

 {
   bootstrapTools = import <nix/fetchurl.nix> {

I wonder if the default should be changed.

[eqyiel]

Here's another workaround: https://stackoverflow.com/a/49405605

QEMU and binfmt support light the way :)

microsoft/WSL#2468 (comment)

After reading that the WSLInterop between WSL and Windows processes used binfmt, I was tinkering with QEMU to try some ARM development, and incidentally discovered how to get 32-bit support working.

Edit: requires "Fall Creators Update", 1709, build 16299 or newer

Install qemu and binfmt config:

sudo apt install qemu-user-static
sudo update-binfmts --install i386 /usr/bin/qemu-i386-static --magic '\x7fELF\x01\x01\x01\x03\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x03\x00\x01\x00\x00\x00' --mask '\xff\xff\xff\xff\xff\xff\xff\xfc\xff\xff\xff\xff\xff\xff\xff\xff\xf8\xff\xff\xff\xff\xff\xff\xff'

You'll need to reactivate binfmt support every time you start WSL:

sudo service binfmt-support start

At this point, Nix should be able to use the i686 bootstrap tools.

[dtzWill]

Neat! Glad the musl tools helped, but the other approach might be preferred in some situations and is interesting regardless! 👍