fix(hash): recreate container on project config content change

[idsulik]

What I did
Fixed hash.ServiceHash() to support config content change

Related issue
#11900

[ndeloof]

While I understand the intent, I don't like we get the config content added into the service hash. This also only makes sense as the config content is inlined.
I wonder we could rely on a label to track the config state by the time it was created : com.docker.compose.config.name=<config hash>.

[idsulik]

I don't like we get the config content added into the service hash

why?

I wonder we could rely on a label to track the config state by the time it was created : com.docker.compose.config.name=<config hash>.

I don't get the idea, time it was created - do you mean config file? or docker-compose.yaml? but docker-compose can refer to external config file

[ndeloof]

time container was created, so we can check it needs to be recreated if current config doesn't match

[ndeloof]

Some poins:

1. At some point we will have to do the same for secrets (same constraints) and networks/volumes (which can get labeled with a has). Solution we adopt here should be extensible to allow this in the future
2. Configs set by file are implemented by a bind mount but this may change in near future (see Enable configs.file's on remote docker hosts #11871) so we also need to consider those
3. There's no technical reason docker engine can't offer secrets/configs natively, just this is guarded by Swarm mode, but AFAIK some discussion happened to get them also available in standalone mode. I can't tell if/when this would take place, but preferably the logic here should consider this may happen
4. Last but not least, I'd prefer we don't mix service config hash with resources it depends on, so my suggestion to introduce an additional label com.docker.compose.config.xx=hash that we can use to track this relation, and need to recreate container, without the need to change the service hash computation (which as impact on existing installations)

[idsulik]

@ndeloof thanks for the details. pushed changes:

1. reverted old changes
2. added new func ServiceDependenciesHash and label

// ConfigHashDependenciesLabel stores configuration hash for a compose service dependencies
ConfigHashDependenciesLabel = "com.docker.compose.config-hash-dependencies"

Let me know if you have better idea for the label name, because I'm not fully satisfied with my name)

[codecov]

Codecov Report

Attention: Patch coverage is 59.77011% with 70 lines in your changes missing coverage. Please review.

Project coverage is 51.69%. Comparing base (49575ef) to head (7c3c8d2).
Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/utils/tar.go	58.22%	22 Missing and 11 partials ⚠️
pkg/compose/convergence.go	32.00%	10 Missing and 7 partials ⚠️
pkg/compose/hash.go	71.42%	11 Missing and 3 partials ⚠️
pkg/compose/create.go	68.42%	4 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #11931      +/-   ##
==========================================
- Coverage   51.78%   51.69%   -0.10%     
==========================================
  Files         156      157       +1     
  Lines       15697    15800     +103     
==========================================
+ Hits         8129     8168      +39     
- Misses       6763     6807      +44     
- Partials      805      825      +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ndeloof]

I'd prefer we have one label per config/secret mount, so it makes it easier to track|debug changes and container being recreated.
Also need to consider config can be mounted from docker host, i.e. file is not available for compose to compute hash, and then must be excluded from label / no label created. createTarForConfig could return ErrNotFound and we would ignore it for this specific usage

[idsulik]

do you mean something like that:

com.docker.compose.service.configs-hash-{configName}={hash}
com.docker.compose.service.configs-hash-{serviceName}={hash}

?

[ndeloof]

indeed, or maybe, to follow the dot-notation style used for labels, com.docker.compose.service.configs.{configName}.hash

[idsulik]

It seems to me that this will complicate the logic, first you need to generate a hash for each item separately, then you need to go through all labels whose names start with “com.docker.compose.service.configs.” to check if the hash has changed.

[ndeloof]

Doesn't look such a pain to me, as this would allow to trace reason we recreate a container, and make it easier to diagnose potential regressions (this sometimes happened :P)

for c := range service.Configs {
  hash := labels["com.docker.compose.configs."+c+".hash"]
  expected := ConfigHash(project.Configs[c]
  if hash := expected {
    log.Debug("container has to be recreated after config %s has been updated", c)
    return DIVERGED
  }
}

[idsulik]

@ndeloof, but we don't have a config/service name that we can use to create the label name. I pushed changes to create a hash of the config/secret of each service so it'll be easy to figure out what service's config/secret caused the change

[ndeloof]

nit: make it simpler as return b, err

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[apollo13]

This would be a really great addition, anything I can help out with?

[idsulik]

This would be a really great addition, anything I can help out with?

no, thank you, waiting for @ndeloof response

[apollo13]

@idsulik Maybe it might make sense to fix the failing linting checks before another round of reviews?

[idsulik]

@ndeloof could you please check it?

[idsulik]

@glours hi! Maybe you can check it? otherwise, it can be stuck here for a long time