[BUG] docker compose up does not recreate containers when named volume configuration is changed

[NReilingh]

Description

Docker compose up is supposed to observe the state of running containers, compare this to the service definition, and recreate containers that need to be converged to the definition.

The options --force-recreate and --no-recreate are meant to override this behavior, so they are not specified when you want containers only to be recreated if a change is needed to bring them in line with the service definition.

When a service definition references a named volume defined in the top level element, currently a change to that volume configuration does not result in existing containers using that named volume being recreated.

Steps To Reproduce

This is tested on Docker Desktop Mac (Apple Silicon)

1. With a compose.yml like the following

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - changer-vol:/changervol

volumes:
  changer-vol:
    name: first

Run docker compose up

2. Then ctrl-c to stop the running container. Observe that a volume named first is mounted to the container
3. Edit the compose file to change the last line to name: second
4. Run docker compose up again

Expected behavior is that the changer container is recreated so that it has a volume named second mounted.

Actual behavior is that the container is not recreated, and still has the first volume mounted.

Compose Version

Docker Compose version v2.12.2

Docker Environment

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.12.2)
  dev: Docker Dev Environments (Docker Inc., v0.0.3)
  extension: Manages Docker extensions (Docker Inc., v0.2.13)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.21.0)

Server:
 Containers: 12
  Running: 2
  Paused: 0
  Stopped: 10
 Images: 33
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1c90a442489720eec95342e1789ee8a5e1b9536f
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.841GiB
 Name: docker-desktop
 ID: RM2G:OFRZ:AO72:RUPG:EMGW:6DOJ:UVW5:OEVK:5B3B:DF2L:AXD7:5VIA
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[ndeloof]

Nice catch.
Compose indeed check service configuration has changed (using a hash) to detect need to recreate, but doesn't check the networks or volumes mounts match the global project configuration.

[ndeloof]

Actually, while I expected we did, volumes don't get re-created on configuration change - this involves a risk to loose data, so would at least require an explicit flag or user to confirm volume replacement.

[NReilingh]

@ndeloof Hm, that doesn't seem consistent with what I'm experiencing.

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - changer-vol:/changervol

volumes:
  changer-vol:
    name: first
  changee-vol:
    name: second

If I docker compose up -d this configuration, then the first volume is created. Then if I change line 6 from changer to changee and re-run docker compose up -d, the container is recreated using the second volume, which is created. first is not deleted, but persists on the host.

[ndeloof]

@NReilingh because you change volume name (which is equivalent to defining another volume). If you change another configuration attribute in a volume, this won't get detected.

[NReilingh]

@ndeloof I might just not be getting this, but isn't that consistent with my initial issue? The only change was the name attribute, so the original volume should not have been recreated (causing data loss), but the container should have been recreated.

Initial:

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - changer-vol:/changervol

volumes:
  changer-vol:
    name: first

Change causes container re-creation:

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - changed-vol:/changervol            # <- Modified

volumes:
  changed-vol:                             # <- Modified
    name: second                           # <- Modified

Change does not cause container re-creation:

services:
  changer:
    image: ealen/echo-server
    container_name: changer-container
    volumes:
      - changer-vol:/changervol

volumes:
  changer-vol:
    name: second                           # <- Modified

[ndeloof]

The only change was the name attribute, so the original volume should not have been recreated (causing data loss)

name is not just a configuration attribute, but actually the unique ID for a volume, changing this means you ask Compose to switch to a distinct volume. basically

services:
  changer:
    volumes:
      - foo:/path

volumes:
  foo:
    name: bar

is equivalent to:

services:
  changer:
    volumes:
      - bar:/path

volumes:
  bar: {}

name should be used to target an existing volume you want to mount, this has no real benefits if you want compose to manage volume lifecycle for you.

As any attribute under services definition is updated, the configuration hash changes and compose detect container needs to be recreated. But using the volume.name attribute to change the volume ID you made this invisible to service configuration - this is something we must fix.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[ndeloof]

also see #9203 for a comparable issue, which comes without the risk we delete user data while re-creating a volume after configuration changed!