Docker Engine 20.10 for the DockerMsftProvider

[TBBle]

The DockerMsftProvider OneGet provider's metadata at https://dockermsft.azureedge.net/dockercontainer/DockerMsftIndex.json does not yet have any 20.10.x releases, even though https://github.com/moby/moby is up to 20.10.5, and Docker Desktop for Windows 3.2.1 has the same.

It's not clear who maintains this metadata and builds these images; the last update was for 19.03.14 in December 2020, and there has been an upstream 19.03.15 security update since for that branch since then. https://github.com/MicrosoftDocs/Virtualization-Documentation-Private/issues/1559 says that this is an MS-maintained and validated build of Docker Engine, but I can't find anything else publicly about it.

Sadly, DockerProvider is even further behind, so that's not a viable alternative.

[james-nesbitt]

The windows engine is not managed through the moby project.

The engine was updated to a 20.10 stream last Friday as a part of the MirantisEngine 20.10 release. You should be able to install using OneGet at this time.

[TBBle]

Yup, I can see 20.10.0 in the DockerMsftIndex.json now.

The windows engine is not managed through the moby project.

Does this mean that the version numbers seen there will not match the version numbers from moby/moby? i.e. is 20.10.0 from DockerMsftProvider actually including changes seen in later releases of moby/moby 20.10.x? Or are they totally disjoint, and I need to track bugs in DockerMsftProvider's Docker Engine separately from those in moby/moby.

One one hand, it'd be nice if DockerMsftProvider pulled, e.g. moby/moby#41430 and moby/moby#41636, into its 20.10.x, the latter of which I'm waiting on the moby/moby 21.x release series to use.

On the other hand, it'll be a pain if I need to tell colleagues that there are bugs that will affect their Windows 10 development machines when they are testing code, but it'll be fine on the Windows Server boxes in the build farm. (Also, I'll need to make sure we're not adding Windows 10 machines to the build farm...) And vice-versa, that would doubles the release trains upon which I need to land BuildKit (and dependent updates) before I can actually start using it for real projects.

Either way, if the version numbers aren't matching moby/moby, it'd be nice to have a changelog or release notes for this.

[weijuans-msft]

@james-nesbitt can you answer the questions from @TBBle "Does this mean that the version numbers seen there will not match the version numbers from moby/moby? i.e. is 20.10.0 from DockerMsftProvider actually including changes seen in later releases of moby/moby 20.10.x?"

On the other hand, I was able to update to the latest release 20.10.0 on my Windows Server 2022 Preview Azure VM:). Thanks @james-nesbitt and team.

[weijuans-msft]

@TBBle Forgot to mention, Docker Inc. no longer maintains that DockerProvider. For Windows customers, we have been suggesting customers to use the instructions at our doc page Set up your environment and use the DockerMSFTProvider.

[TBBle]

@weijuans-msft Thank you for the update on DockerProvider.

@james-nesbitt I am assuming https://docs.mirantis.com/containers/v3.1/dockeree-release-notes/mcr-relnotes.html is the correct release notes feed for the container engine coming via DockerMsftIndex.json now? It certainly aligns 20.10.0 and 19.03.14 as the latest releases for those two channels.

A quick check shows nothing in the Moby 20.10.x releases after 20.10.0 appears in the MCR Release Notes for 20.10.0. I didn't check the converse though, but I assume there are fixes/changes/improvements in MCR 20.10.0 over Moby 20.10.0 based on seeing non-GitHub issue references in the MCR release notes.

[james-nesbitt]

Hi guys, sorry for the delay in response.

Mirantis efforts on MCR (engine) and the servicing of the MSFT OneGet Docker packages will be attempt to maintain feature parity with the upstream Docker releases and will try to follow the same numbering. There may be some cherry-picking of unreleased commits, that would show up as a misalignment, but the over the long term all of the source will line up.
Of note to you, there will soon be an incoming feature bump right to 20.10.4.

The release delay comes down more to Mirantis release schedules and timing than any strategic decision to diverge the enterprise product from the upstream community edition.

[mthalman]

Will the feature bump to 20.10.4 include moby/moby#41636?

[weijuans-msft]

@james-nesbitt please chime in.

[james-nesbitt]

We build the windows releases from the named branches. AFAIK none of the 127GB default sandbox size changes have made it into the 20.10 branch from which we build.
See https://github.com/moby/moby/blob/20.10/daemon/graphdriver/windows/windows.go

I'll go to the engineering team and ask for more information.

[james-nesbitt]

To expand further, the 127GB change looks like it hasn't been included in the 20.10.5 release, as it is still in master: moby/moby@v20.10.5...master

[TBBle]

That's correct, moby/moby#41636 has not been backported to 20.10.x upstream.

I understood #99 (comment) was a request for it to be backported for the Mirantis-serviced DockerMSFTProvider build specifically.

[weijuans-msft]

Quick update: I will be working with our friends at Mirantis (James, Uday etc) to coordinate future releases and provide a better experiences. There is no action item specific here so I intend to close this in a couple days, allowing anyone else to comment if any.

[TBBle]

Looks like we're done here. 20.10.4 is available, and future releases will hopefully have better visibility. Thank you @james-nesbitt and @weijuans-msft .