Shared volume mode/propagation not working

[nemchik]

Description

Attempting to use :shared on a volume instead instead creates a volume with mode rw and propagation rprivate

Steps to reproduce the issue:
docker-compose.yml file

services:
  test:
    container_name: test
    image: alpine
    volumes:
    - /mnt/test:/mnt/test:shared

docker compose config output

name: test
services:
  test:
    container_name: test
    image: alpine
    networks:
      default: null
    volumes:
    - type: bind
      source: /mnt/test
      target: /mnt/test
      bind:
        propagation: shared
        create_host_path: true
networks:
  default:
    name: test_default

run docker compose up -d

docker inspect test output

[
    {
        "Id": "6d8ea8b6e36c11b7623c1313d0deb04284c3a2c14beca35dbd72635d80f2988b",
        "Created": "2022-04-11T23:54:08.248168714Z",
        "Path": "/bin/sh",
        "Args": [],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-04-11T23:54:10.105565887Z",
            "FinishedAt": "2022-04-11T23:54:10.123330891Z"
        },
        "Image": "sha256:0ac33e5f5afa79e084075e8698a22d574816eea8d7b7d480586835657c3e1c8b",
        "ResolvConfPath": "/var/lib/docker/containers/6d8ea8b6e36c11b7623c1313d0deb04284c3a2c14beca35dbd72635d80f2988b/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/6d8ea8b6e36c11b7623c1313d0deb04284c3a2c14beca35dbd72635d80f2988b/hostname",
        "HostsPath": "/var/lib/docker/containers/6d8ea8b6e36c11b7623c1313d0deb04284c3a2c14beca35dbd72635d80f2988b/hosts",
        "LogPath": "/var/lib/docker/containers/6d8ea8b6e36c11b7623c1313d0deb04284c3a2c14beca35dbd72635d80f2988b/6d8ea8b6e36c11b7623c1313d0deb04284c3a2c14beca35dbd72635d80f2988b-json.log",
        "Name": "/test",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/mnt/test:/mnt/test:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "test_default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/cadf74c817fdd1e9faa578088667d255f1796a5dfacaed1040b3742c1114f3be-init/diff:/var/lib/docker/overlay2/638430a0acc2601a319ee03ae839448eb5b328f3f12465b1855ea63dadb5a96d/diff",
                "MergedDir": "/var/lib/docker/overlay2/cadf74c817fdd1e9faa578088667d255f1796a5dfacaed1040b3742c1114f3be/merged",
                "UpperDir": "/var/lib/docker/overlay2/cadf74c817fdd1e9faa578088667d255f1796a5dfacaed1040b3742c1114f3be/diff",
                "WorkDir": "/var/lib/docker/overlay2/cadf74c817fdd1e9faa578088667d255f1796a5dfacaed1040b3742c1114f3be/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/mnt/test",
                "Destination": "/mnt/test",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "6d8ea8b6e36c",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh"
            ],
            "Image": "alpine",
            "Volumes": {
                "/mnt/test": {}
            },
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "8ec02ba42ee5418fd4260161447bea91ff2068c920102ebcc03eb43ad06fb362",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.depends_on": "",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "test",
                "com.docker.compose.project.config_files": "/home/castor/test/docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/castor/test",
                "com.docker.compose.service": "test",
                "com.docker.compose.version": "2.4.1"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "b348dda01772880dde375358a6689bb93fedaa51efeb0fa4ab3bc855205f1610",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/b348dda01772",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "test_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "test",
                        "test",
                        "6d8ea8b6e36c"
                    ],
                    "NetworkID": "8aa5fa473b730d00498166d0bfdf36f19e7c1de424d53ee32e4a39de03b14497",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

Describe the results you received:

Shown above in the docker inspect test output, specifically this section:

        "Mounts": [
            {
                "Type": "bind",
                "Source": "/mnt/test",
                "Destination": "/mnt/test",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

Describe the results you expected:

The container should be created with a shared volume

        "Mounts": [
            {
                "Type": "bind",
                "Source": "/mnt/test",
                "Destination": "/mnt/test",
                "Mode": "shared",
                "RW": true,
                "Propagation": "shared"
            }
        ],

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.4.1

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.1-docker)
  compose: Docker Compose (Docker Inc., v2.4.1)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 10
  Running: 9
  Paused: 0
  Stopped: 1
 Images: 15
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-107-generic
 Operating System: Ubuntu 20.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.771GiB
 Name: vmi439163.contaboserver.net
 ID: 7T6T:3I4A:7YWK:7LBC:WFRY:4T7G:AXO5:QFWT:E6JL:WFXU:LSRX:LVB2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

Using compose v1 here are the differences:

/usr/local/bin/docker-compose-v1 version output

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.8.10
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020

/usr/local/bin/docker-compose-v1 config output

services:
  test:
    container_name: test
    image: alpine
    volumes:
    - /mnt/test:/mnt/test:shared
version: '3.9'

run /usr/local/bin/docker-compose-v1 up -d

docker inspect test output

[
    {
        "Id": "853fd32f0ad83bbcade7a489228508cdb518271adc7b7446effe0c3669867ea7",
        "Created": "2022-04-12T00:08:44.211758995Z",
        "Path": "/bin/sh",
        "Args": [],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-04-12T00:08:45.496652188Z",
            "FinishedAt": "2022-04-12T00:08:45.511877452Z"
        },
        "Image": "sha256:0ac33e5f5afa79e084075e8698a22d574816eea8d7b7d480586835657c3e1c8b",
        "ResolvConfPath": "/var/lib/docker/containers/853fd32f0ad83bbcade7a489228508cdb518271adc7b7446effe0c3669867ea7/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/853fd32f0ad83bbcade7a489228508cdb518271adc7b7446effe0c3669867ea7/hostname",
        "HostsPath": "/var/lib/docker/containers/853fd32f0ad83bbcade7a489228508cdb518271adc7b7446effe0c3669867ea7/hosts",
        "LogPath": "/var/lib/docker/containers/853fd32f0ad83bbcade7a489228508cdb518271adc7b7446effe0c3669867ea7/853fd32f0ad83bbcade7a489228508cdb518271adc7b7446effe0c3669867ea7-json.log",
        "Name": "/test",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/mnt/test:/mnt/test:shared"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "test_default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/98e6401683210e4387b2e1378b0f1d03522102bdd9ea38565a31eca3c214ee62-init/diff:/var/lib/docker/overlay2/638430a0acc2601a319ee03ae839448eb5b328f3f12465b1855ea63dadb5a96d/diff",
                "MergedDir": "/var/lib/docker/overlay2/98e6401683210e4387b2e1378b0f1d03522102bdd9ea38565a31eca3c214ee62/merged",
                "UpperDir": "/var/lib/docker/overlay2/98e6401683210e4387b2e1378b0f1d03522102bdd9ea38565a31eca3c214ee62/diff",
                "WorkDir": "/var/lib/docker/overlay2/98e6401683210e4387b2e1378b0f1d03522102bdd9ea38565a31eca3c214ee62/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/mnt/test",
                "Destination": "/mnt/test",
                "Mode": "shared",
                "RW": true,
                "Propagation": "shared"
            }
        ],
        "Config": {
            "Hostname": "853fd32f0ad8",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh"
            ],
            "Image": "alpine",
            "Volumes": {
                "/mnt/test": {}
            },
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "9d2afe52cfd02d462058a7f438ac9606bfd0f82b56b3e9d4f647e34235f3c20a",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "test",
                "com.docker.compose.project.config_files": "docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/castor/test",
                "com.docker.compose.service": "test",
                "com.docker.compose.version": "1.29.2"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "e997ba118c274d5fbb657b42426eaf90065ef16342907f2a06e71b155fb9673b",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/e997ba118c27",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "test_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "test",
                        "853fd32f0ad8"
                    ],
                    "NetworkID": "25fed5eb46b3e151cb497fbf7676204c4cc3e6498ee425ce41ed91614e93fb43",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

[ndeloof]

Have you tried using docker compose V2 to confirm this issue also applies to the new codebase?

[nemchik]

Have you tried using docker compose V2 to confirm this issue also applies to the new codebase?

Yes. Above I've detailed the results of compose v2.4.1 (incorrect behavior) and compose v1.29.2 (correct behavior).

I have not set the version in the docker-compose.yml file if that's what you're asking.

[ndeloof]

root cause is we use the bind API for such a volume, due to distinct behavior, while the API reports this bind mount using the mount API. Using compose-spec/compose-go#248 we will be able to pass the relevant volume string to the API.

[nemchik]

Does this require a new release of compose v2? Or is this something I can test with v2.4.1 now that compose-spec/compose-go#248 is merged?

[ndeloof]

still need to bump compose-go dependency into docker/compose, adapt code to use this improvement and release a new 2.4.x

[zbyte64]

I think I am still seeing this issue.

> docker compose version
Docker Compose version v2.40.1

> cat docker-compose.yaml
...
volumes:
      - type: volume
        source: s3data
        target: /opt/s3fs/bucket
        bind:
          propagation: shared
...

> docker inspect <container>
...
"Mounts": [
            {
                "Type": "volume",
                "Name": "s3data",
                "Source": "/var/lib/docker/volumes/s3data/_data",
                "Destination": "/opt/s3fs/bucket",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],
...

[ndeloof]

@zbyte64 bind is only relevant with type: bind, otherwise ignored (could we reported as an error)