Building both Linux and Windows images in parallel on Windows

[0x53A]

Tell us about your request
I want to be able to build both Windows and Linux images on the same machine in parallel

Which service(s) is this request for?
docker build / Docker Desktop Windows

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

We have an existing CI infrastructure with Windows machines.

We started our journey to the cloud by packaging our services up into a big, chunky Windows Server Core container.

Now we are migrating some services to .NET Core and Linux, but don't want to add a dedicated linux build server.

.NET Core does support cross compiling for Linux from Windows, so we just need to package it into a linux image.

Are you currently working around the issue?
At the moment we are using LCOW. The main issue is docker/for-win#2623.

Additional context
docker/for-win#6470

LCOW is somewhat lingering in limbo.

WSL2 is a replacement for the MobyLinux VM, but NOT for LCOW because you need to explicitly switch the whole docker daemon between Windows and Linux.

[stephen-turner]

This issue and #79 are the same (or sufficiently the same that they can be combined into one feature request). I know this one was earlier, but #79 has more thumbs up, so I'm going to preserve that one and close this one as a dupe.

[conioh]

@0x53A, don't use Docker Inc.'s lame "Docker Desktop". Use plain-old Docker and everything works. I saw a few people have been asking about it so I put the details in https://github.com/conioh/Docker-Windows-Linux so I won't have to repeat myself. I hope this helps you. It's not a perfect replacement, but it goes 99% of the way and that's what I'm using.

[sirlatrom]

@0x53A, don't use Docker Inc.'s lame "Docker Desktop". Use plain-old Docker and everything works. I saw a few people have been asking about it so I put the details in https://github.com/conioh/Docker-Windows-Linux so I won't have to repeat myself. I hope this helps you. It's not a perfect replacement, but it goes 99% of the way and that's what I'm using.

From that repo:

Create a /etc/docker/daemon.json and put in something like:

{
    "debug": true,
    "hosts": ["unix://", "tcp://0.0.0.0:2375"],
    "experimental": true
}

That repo looks like it suggests exposing the Docker socket unauthenticated on port 2375 which is considered harmful. If you want to do this in a safe manner, you are recommended to protect the Docker daemon socket.

[conioh]

That repo looks like it suggests exposing the Docker socket unauthenticated on port 2375 which is considered harmful. If you want to do this in a safe manner, you are recommended to protect the Docker daemon socket.

@sirlatrom: You are aware that this machine is a VM connected through a NAT to the host, right? The network it's "exposed" to is not the physical network to which the host is connected.

[fredrikhr]

@conioh well, the deamon configuration still runs the docker deamon on a TCP port that is open to everyone on your network (0.0.0.0), this is not the network configuration between the docker host and the container!

Personally I use npipe:////./pipe/docker_engine and npipe:////./pipe/docker_engine_wsl2 instead. That will uses named pipes (npipe) instead and thus automatically protects the docker deamon because windows will do named pipe impersonation and thus respect the group configuration of the deamon.

[fredrikhr]

@conioh Sorry, I misunderstood. You were referring to the docker daemon running inside WSL2. And you're absolutely correct: The network of the WSL is only visible to the Windows Host and thus using the network is actually safe in this instance.

[conioh]

@fredrikhr, that's right.

There can still be danger if you're running unsafe or vulnerable code on your host (and let outsiders access it) or if you change the WSL network to be bridged, etc.

If you're doing any of those things I assume you know what you're doing and don't need me to tell you how to do it right, or how to run apt to install Docker. In that case I assume you know what to configure and how.

[PavelSosin-320]

It is possible to run several docker providers on the same Windows Host and expose every docker as TCP entry-point without conflicts if different ports are assigned. Running docker on the hardened Linux. But it doesn't make the parallel building useful because the docker always keeps in mind the last created image when works with the registry. 2 recent built images will be very confusing for docker. Docker works with the registry in a sequential manner: build - tag - push. The docker concept says that local image storage inside Docker daemon FS tree is temporary, short-time persistent storage before transfer to the registry.
Even if the parallel build is imaginable, parallel tag or push is a nightmare.

[PavelSosin-320]

There is the rich library of Linux Dotnet base container images and these base images should be evaluated for Dotnet Docker applications development because they share the same OS layers, i.e. optimize overall memory consumption and startup time. They provide Dotnet compatible runtime up to Dotnet 3.1 without any underlying Windows layers.
On the other hand, I suppose, it solves the issue. It's very hard to create Dockerfiles that reuse something even for base images that use different Linux flavors: Debian, Ubuntu, Fedora, because every Linux flavor has its own package manager, own Repository, and own configuration utility. i afraid, the same Dockerfile which can be used for building more than exactly 1 image is impossible for today. The Dockerfiles for Linux-based and Windows-based images differ already in the 1st line because Microsoft doesn't use Dockerhub to publish its images.

[thomasoatman]

@conioh Thank you so much. I was developing a build system based on a number of containers with the expectation of running Linux and Windows either back-to-back or async. It was working when I started and somewhere along the installed updates, it stopped. I thought I had done something wrong.
I got pulled off this project back in April/May and had just started back on it when your post came up.
I am SOOOOOO thankful. I have not fully tested my actual builds . . . I spent the last day or two perfecting my VM image and documenting ALL of the steps into one cross-referenced doc (including the Ubuntu 20.04 hurdles)
So far, it is looking amazing. Thank you again!
Docker-Windows-Linux-B2.pdf

[whitmell]

@thomasoatman thanks for consolidating @conioh 's instructions into one easy to follow doc. I followed your instructions and I couldn't get that setup to work. The docker daemon within WSL (ubuntu 18.04) wouldn't start. The only way I could get it to work was to modify C:\ProgramData\docker\config\daemon.json on the host to include the host ""tcp://0.0.0.0:2375", but if I'm understanding correctly this just points the WSL docker back to the host daemon, which I thought was not the intent of this setup. In my current configuration, I'm back to using Linux Containers for Windows to run linux containers from the host daemon. From what I'm reading, docker daemon can't run on WSL1. I'm using Windows Server 2019 as the host and don't have the ability to use WSL2. Any suggestions?

[PavelSosin-320]

@thomasoatman I installed the native docker CE on the top of CentOS7 WSL (from Microsoft store) distro and it took less than 2 hours including the original Docker for CentOS 7 installation guide reading. Then I exposed Docke's socket using sockat to port 2375 and it works. It starts about 10 times faster than Docker Desktop.
I'm trying to continue with Podman now because Docker's support is revoked for the further versions of RHEL and CentOS

[conioh]

@whitmell, indeed there's no WSL2 for Server 2019 so the whole thing is irrelevant. You can either use a full-fledged VM (e.g. MobyVM) for your Linux containers, or you can still use LCOW. If you don't mind it being experimental and unsupported using LCOW seems a lot easier to me.

[whitmell]

@conioh thanks. LCOW not being supported is a pain, and I actually had to roll back my version of docker to an earlier 2020 version for my linux containers to function again on Windows Server 2019. This is all relatively temporary as most of my future work will be on linux servers, but an annoyance in the meantime. Thanks for putting together that great guide!