Add an opt-out for iptables 'raw' rules #49621

robmry · 2025-03-10T16:01:07Z

- What I did

fix Failed to set up container networking 28.0.1 (module IP_NF_RAW dependency) #49557

For kernels that don't have CONFIG_IP_NF_RAW, if the env var DOCKER_INSECURE_NO_IPTABLES_RAW is set to "1", don't try to create raw rules.

Warning: When the environment variable is set, direct routing to published ports is possible from other hosts on the local network, even if the port is published to a loopback address. It un-does some of the security hardening described at https://www.docker.com/blog/docker-engine-28-hardening-container-networking-by-default/

- How I did it

The env var is DOCKER_INSECURE_NO_IPTABLES_RAW ... because that's why the workaround is needed. Alternatively, it could be called something like DOCKER_INSECURE_ALLOW_DIRECT_ROUTING ... because that's currently the effect. Then it'd need to do the same thing for an nftables/firewalld implementation.

If we want to add a more "feature-y" way to allow direct routing at some point - it should be via a new "gateway mode" with well defined semantics, something less drastic than nat-unprotected (allowing access from remote hosts, but only to published ports, and maybe not to ports published to 127.0.0.1). That'd work per network rather than globally, and it'd need some different regression testing and more documentation.

So, I went for this simple workaround for kernels without the required module.

- How to verify it

New integration test.

Also, checked a container with host port mapping started on a host without CONFIG_IP_NF_RAW ...

# cat /etc/systemd/system/docker.service.d/insecure_direct_routing.conf
[Service]
Environment="DOCKER_INSECURE_NO_IPTABLES_RAW=1"
# systemctl daemon-reload
# systemctl restart docker
# docker run --rm -ti -p 127.0.0.1:8080:80 alpine

In docker's log ...

Mar 10 15:38:50 debian2 dockerd[7232]: time="2025-03-10T15:38:50.830162607Z" level=debug msg="DOCKER_INSECURE_NO_IPTABLES_RAW=1 - skipping raw rules" eid=3c070182713114fcf3b58f606cd0d2dc773516360c0bd9eed60bb4899d90d467 ep=elated_keldysh net=bridge nid=b30950846c3a84d5e17e8cf0406ff435a76323e87d41d3fdf47262032e670af6
Mar 10 15:38:50 debian2 dockerd[7232]: time="2025-03-10T15:38:50.830260399Z" level=debug msg="DOCKER_INSECURE_NO_IPTABLES_RAW=1 - skipping raw rules" eid=3c070182713114fcf3b58f606cd0d2dc773516360c0bd9eed60bb4899d90d467 ep=elated_keldysh net=bridge nid=b30950846c3a84d5e17e8cf0406ff435a76323e87d41d3fdf47262032e670af6

(And, no rules created in the iptables/ip6tables raw table.)

- Human readable description for the release notes

Add the environment variable `DOCKER_INSECURE_NO_IPTABLES_RAW=1` to allow Docker to run on systems where the Linux kernel can't provide `CONFIG_IP_NF_RAW` support. When enabled, Docker will not create rules in the iptables `raw` table. Warning: This is not recommended for production environments as it reduces security by allowing other hosts on the local network to route to ports published to host addresses, even when they are published to `127.0.0.1.` This option bypasses some of the security hardening introduced in Docker Engine 28.0.0.

thaJeztah · 2025-03-10T16:12:24Z

Perhaps something we should add to the warnings returned by docker info, so that it's visible to the user that these are disabled.

Not sure how easy it would be to punch through this option all the way from the daemon config, but if that's not trivial (likely isn't?) we could add the same check for the env-var somewhere in the daemon.systeminfo (or whatever a good place would be)

robmry · 2025-03-10T16:26:08Z

Perhaps something we should add to the warnings returned by docker info, so that it's visible to the user that these are disabled.

Sure, how does this look? ... could add some words about the implications, but that could get a bit too wordy?

# docker info
Client:
 Version:    28.0.1
 [...]
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 [...]
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

[DEPRECATION NOTICE]: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
In future versions this will be a hard failure preventing the daemon from starting! Learn more at: https://docs.docker.com/go/api-security/
WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW=1

(Maybe worth noting, I think this will go-away once we've switched to native nftables.)

thaJeztah · 2025-03-10T18:18:18Z

Sure, how does this look? ... could add some words about the implications, but that could get a bit too wordy?

Probably fine; if we need more, we could of course add a https://docs.docker.com/go/.... link, but if it's temporary, and an explicit opt-in, it's probably fine to keep it basic. Unless you have a very brief description, but otherwise, perhaps just leave it as this.

(Having it in docker info can help us as well when triaging bug-reports if the user posts the output in the ticket)

thaJeztah · 2025-03-10T18:21:44Z

daemon/info_unix.go

 	}
+	// Env-var belonging to the bridge driver, disables use of the iptables "raw" table.
+	if os.Getenv("DOCKER_INSECURE_NO_IPTABLES_RAW") == "1" {
+		v.Warnings = append(v.Warnings, "WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW=1")


If we do want to add some words around it, perhaps something like;

Suggested change

v.Warnings = append(v.Warnings, "WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW=1")

v.Warnings = append(v.Warnings, "WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW is set")

Suggested change

v.Warnings = append(v.Warnings, "WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW=1")

v.Warnings = append(v.Warnings, "WARNING: raw iptables rules are skipped because DOCKER_INSECURE_NO_IPTABLES_RAW is set")

The 'DOCKER_INSECURE_NO_IPTABLES_RAW is set` may be slightly nicer to indicate what's the case, but not sure about the "raw iptable rules are skipped", not sure if that's providing much context on its own.

Ok, yes - I'll change it to "is set".

thaJeztah · 2025-03-10T18:22:36Z

integration/networking/port_mapping_linux_test.go

+	"gotest.tools/v3/golden"
+
 	containertypes "github.com/docker/docker/api/types/container"


Micro-nit (if you'll be pushing again); looks like this one ended up separate from the other imports below

Doh! Thank you ... something in my GoLand setup does that, maybe gofumpt. It's very annoying, but haven't tracked it down yet.

Yeah, possibly it's using a different convention (which we possibly should adopt) where imports are in 3 blocks instead of 2;

[go stdlib imports] [external imports] [local imports]

That's kinda ok, but I see it also more often result in accidental splitting up blocks even more. I think there's a linter for that though (still to look into)

For kernels that don't have CONFIG_IP_NF_RAW, if the env var DOCKER_INSECURE_NO_IPTABLES_RAW is set to "1", don't try to create raw rules. This means direct routing to published ports is possible from other hosts on the local network, even if the port is published to a loopback address. Signed-off-by: Rob Murray <[email protected]>

thaJeztah · 2025-03-10T18:43:52Z

libnetwork/drivers/bridge/port_mapping_linux.go


+func rawRulesDisabled(ctx context.Context) bool {
+	if os.Getenv("DOCKER_INSECURE_NO_IPTABLES_RAW") == "1" {
+		log.G(ctx).Debug("DOCKER_INSECURE_NO_IPTABLES_RAW=1 - skipping raw rules")


Is this logged once, or for every container? If it's once, then Warn would be fine as well

Every container, on create and delete ... I figured it'd be better to repeat it, so it's there in any log snippets we get.

thaJeztah

code LGTM

I'll leave it to @akerouanton for the networking side of things 🤗

vvoland · 2025-03-13T11:27:52Z

WDYT about changelog:

Add environment variable `DOCKER_INSECURE_NO_IPTABLES_RAW=1` to allow Docker to run on systems where the Linux kernel can't provide `CONFIG_IP_NF_RAW` support. When enabled, Docker will not create rules in the iptables `raw` table. Warning: This is not recommended for production environments as it reduces security by allowing other hosts on the local network to route to ports published to host addresses, even when they are published to `127.0.0.1.` This option bypasses some of the security hardening introduced in Docker Engine 28.0.0.

robmry added this to the 28.0.2 milestone Mar 10, 2025

robmry self-assigned this Mar 10, 2025

robmry force-pushed the skip_raw_rules branch from 96430fb to edd2dc3 Compare March 10, 2025 16:15

robmry force-pushed the skip_raw_rules branch from edd2dc3 to 2958197 Compare March 10, 2025 16:43

robmry marked this pull request as ready for review March 10, 2025 17:31

robmry requested review from akerouanton and thaJeztah March 10, 2025 17:32

thaJeztah reviewed Mar 10, 2025

View reviewed changes

robmry force-pushed the skip_raw_rules branch from 2958197 to cf3e42a Compare March 10, 2025 18:33

thaJeztah reviewed Mar 10, 2025

View reviewed changes

thaJeztah approved these changes Mar 10, 2025

View reviewed changes

akerouanton approved these changes Mar 13, 2025

View reviewed changes

akerouanton merged commit 4ff19b2 into moby:master Mar 13, 2025
152 checks passed

irctrakz mentioned this pull request Mar 17, 2025

Also add DOCKER_INSECURE_NO_IPTABLES_RAW to daemon.json #49651

Closed

jtgasper3 mentioned this pull request Mar 24, 2025

Bump github.com/docker/docker from 28.0.1+incompatible to 28.0.2+incompatible jtgasper3/swarm-visualizer#12

Closed

ota-ri mentioned this pull request Apr 10, 2025

Error starting container xxxxxx: 500 Server Error; iptables v1.8.7 (legacy): can't initialize iptables table `raw' (running the AVP demo on Local Hardware) DensoSVIC/soafee-avp-blueprint#27

Open

robmry mentioned this pull request Apr 11, 2025

Inter host container communication is not working for Publish ports with Flannel VXLAN in Docker V28.0 #49792

Closed

robmry mentioned this pull request Apr 27, 2025

Failed to set up container networking 28.0.1 (module IP_NF_RAW dependency) #49557

Closed

robmry deleted the skip_raw_rules branch May 29, 2025 18:51

	v.Warnings = append(v.Warnings, "WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW=1")
	v.Warnings = append(v.Warnings, "WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW is set")

		"gotest.tools/v3/golden"

		containertypes "github.com/docker/docker/api/types/container"

Add an opt-out for iptables 'raw' rules #49621

Add an opt-out for iptables 'raw' rules #49621

Uh oh!

Conversation

robmry commented Mar 10, 2025 • edited by vvoland Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Mar 10, 2025

Uh oh!

robmry commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Mar 10, 2025

Uh oh!

thaJeztah Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

robmry Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

robmry Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

robmry Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vvoland commented Mar 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

robmry commented Mar 10, 2025 •

edited by vvoland

Loading

robmry commented Mar 10, 2025 •

edited

Loading

thaJeztah Mar 10, 2025 •

edited

Loading