[Bug]: /proc/sys/net/ipv4/ip_forward is no longer writable (regression)

[ktarplee]

I have done the following

• I have searched the existing issues
• If possible, I've reproduced the issue using the 'main' branch of this project

Steps to reproduce

container run --rm -it --cap-add ALL busybox sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

Outputs

sh: line 0: can't create /proc/sys/net/ipv4/ip_forward: Read-only file system

The file is actually writable by root. It just mounted as a read-only mount now (it seems).

I found this because I typically run dagger-engine within an apple container. That worked with 1.1.0 but no longer works with 1.2.0.

My actual use case is more closely

container run --rm -it --cap-add ALL registry.dagger.io/engine:v0.21.7

#!/bin/sh
set -e

cat $0

# cgroup v2: enable nesting
# see https://github.com/moby/moby/blob/38805f20f9bcc5e87869d6c79d432b166e1c88b4/hack/dind#L28
if [ -f /sys/fs/cgroup/cgroup.controllers ]; then
        # move the processes from the root group to the /init group,
        # otherwise writing subtree_control fails with EBUSY.
        # An error during moving non-existent process (i.e., "cat") is ignored.
        mkdir -p /sys/fs/cgroup/init
        xargs -rn1 < /sys/fs/cgroup/cgroup.procs > /sys/fs/cgroup/init/cgroup.procs || :
        # enable controllers
        sed -e 's/ / +/g' -e 's/^/+/' < /sys/fs/cgroup/cgroup.controllers \
                > /sys/fs/cgroup/cgroup.subtree_control
fi

# expect more open files due to per-client SQLite databases
# many systems default to 1024 which is far too low
ulimit -n 1048576 || echo "cannot increase open FDs with ulimit, ignoring"

exec tini -- /usr/local/bin/dagger-engine --config /etc/dagger/engine.toml "$@"
INFO[0000] starting dagger engine version:v0.21.7 v0.21.7 linux/arm64/v8 
INFO[0000] configuring iptables backend: legacy         
INFO[0000] set /usr/bin/iptables -> /usr/bin/xtables-legacy-multi 
INFO[0000] set /usr/bin/iptables-save -> /usr/bin/xtables-legacy-multi 
INFO[0000] set /usr/bin/iptables-restore -> /usr/bin/xtables-legacy-multi 
INFO[0000] set /usr/bin/ip6tables -> /usr/bin/xtables-legacy-multi 
INFO[0000] set /usr/bin/ip6tables-save -> /usr/bin/xtables-legacy-multi 
INFO[0000] set /usr/bin/ip6tables-restore -> /usr/bin/xtables-legacy-multi 
INFO[0000] using ipMasqBackend: iptables                
INFO[0000] detected mtu 1280 via interface eth0         
INFO[2026-07-29T11:10:44Z] engine name: 84c39228-0023-4137-94d2-2d9b84e872d4 
DEBU[2026-07-29T11:10:44Z] creating engine GRPC server                  
DEBU[2026-07-29T11:10:44Z] creating engine lockfile                     
DEBU[2026-07-29T11:10:44Z] creating engine server                       
INFO[2026-07-29T11:10:44Z] auto snapshotter: using overlayfs            
DEBU[2026-07-29T11:10:44Z] could not read "/var/lib/dagger/net/cni" for cleanup: open /var/lib/dagger/net/cni: no such file or directory 
shutting down telemetry...
telemetry shut down complete
dagger-engine: failed to create engine: failed to create network providers: CNI setup error: plugin type="bridge" failed (add): failed to enable forwarding: open /proc/sys/net/ipv4/ip_forward: read-only file system

I like running dagger engine in an apple container because I get more isolation than just running it in podman as a privileged container. --cap ALL has been enough in the past to make this work fine.

Problem description

dagger-engine no longer works in apple containers. Worked in 1.1.0 but not in 1.2.0.

Environment

- OS: 26.5.2
- Xcode: 26.6
- Container: 1.2.0

Code of Conduct

• I agree to follow this project's Code of Conduct

[singhible]

I reproduced this. 1.1.0 writes the file, main at 6e65319 gives the same read only error.

It arrived with the containerization bump from 0.35.0 to 0.40.1. apple/containerization#795 added a default readonlyPaths that includes /proc/sys, which matches runc, so I assume it was on purpose.

What I cannot find is a way to opt out. #1996 put maskedPaths and readonlyPaths on the container config, but nothing sets them from the CLI, and --cap-add ALL does not lift them.

Do you want those exposed as flags on run and create? I can write that. The other option is that the default is wrong here, since each container gets its own VM kernel, but that is a call for you rather than something I would send a PR for.

[ronandi]

Reproduced on 1.2.0 (macOS 26.6, M4 Max). Beyond dagger, this also breaks Kubernetes node images entirely: kindest/node exits at boot (mount: /sys: permission denied in the entrypoint) and rancher/k3s fails in the kubelet (Failed to start ContainerManager: open /proc/sys/vm/overcommit_memory: read-only file system).

Root cause is what @singhible found: containerization#795's defaultReadonlyPaths() (includes /proc/sys) started being enforced with the 0.40.x bump, and while #1996 put readonlyPaths/maskedPaths/sysctls on the container config, nothing in the CLI exposes them, so there's no way out right now.

I've opened #2064 adding --sysctl key=value and --privileged (clears the default masked/readonly paths, grants all caps — same semantics as Docker) to run/create. Verified against 1.2.0: --sysctl net.ipv4.ip_forward=1 lands in the guest, and --privileged gets kindest/node and k3s booting again. Since every container is its own VM here, an explicit privileged flag feels proportionate. Happy to split it up or drop --privileged if you'd rather keep the surface smaller.

[jglogan]

@singhible @ronandi we're working on making maskedPaths and readonlyPaths accessible from the CLI.

[prskr]

I might be wrong but at least from the documentation it does not seem like this was also applied to container machines? Even though there is the same issue?

[adityaramani]

@prskr You are right. We need a follow up PR to add the same functionality to container machine. I will create a new issue for this

[adityaramani]

Created #2077