RFC: Use BuildKit for all images

[slonopotamus]

BuildKit offers much shorter build times

---

This is a somewhat controversial change. On one hand, BuildKit severely improves build times. On the other hand, there are reasons why BuildKit is still not enabled by default in Docker.

Some numbers:

Linux
HDD
Ryzen 3700X
ue4-minimal only (ue4-source is already pre-built)
4.20.3
Flags: --exclude debug --exclude ddc --no-engine --no-full

DOCKER_BUILDKIT=1 1h 6m
DOCKER_BUILDKIT=0 1h 27m

[TBBle]

Since we turn it on magically in some circumstance, it makes sense to me to enable it always.

We lose the ability to diagnose build issues using the intermediate layers as images, but I guess that's mostly a tool we've used on WIndows due to filesize issues, where this won't work until we have BuildKit for Windows Containers anyway.

Docker 21.xx will have BuildKit as default on Linux anyway (when run from the docker CLI), according to docker/cli#3314. With DOCKER_BUILDKIT=1 set, fallback to the classic builder in the case of missing buildx plugin will be blocked, so this change still makes sense to require BuildKit.

For other readers, "reasons why BuildKit is still not enabled by default in Docker": moby/moby#40379 discusses the differences in features and behaviour.

[slonopotamus]

Okay, let's merge then. If we later understand that this makes us/users less happy, we can always revert this.