-
Notifications
You must be signed in to change notification settings - Fork 7.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changing the structure of the section #15164
Changes from 6 commits
268391f
c6abbdc
0173430
d469142
4520880
4392104
18bccf8
7b88125
0c69399
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -1392,11 +1392,14 @@ manuals: | |||||
title: Hello Build | ||||||
- sectiontitle: Buildx | ||||||
section: | ||||||
- path: /build/buildx/ | ||||||
title: Working with Buildx | ||||||
- path: /build/buildx/install/ | ||||||
title: Install Buildx | ||||||
|
||||||
- path: /build/buildx/ | ||||||
title: Working with Buildx | ||||||
- path: /build/buildx/multiple-builders/ | ||||||
title: Using multiple builders | ||||||
- path: /build/buildx/multiarch-images/ | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should be consistent:
Suggested change
Needs to rename There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's true. It's called "multi-arch" in Desktop docs, and in conversation we do say multi-arch all the time. Also, taking for example this sentence in the original content: Just trying to point out that it seems to be a combination of both. Platform is not enough to determine the arch (following the above interpretation): imho multi-architecture being more specific is a better candidate. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hum right, WDYT @tonistiigi @jedevc @thaJeztah? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've noticed this inconsistency as well, agreed it would be good to clarify this, but maybe out of scope for this PR, there's a lot of build docs that mix-and-match between the two, so I'd propose to pick something here that works, and then discuss separately later. Personally, I think multi-platform is better when talking about building images. A couple reasons:
Maybe there are still use-cases for talking about multi-arch though, for example, maybe around QEMU, when we're using linux throughout, but emulating different CPU architectures.
I think this is probably unrelated, I think this refers to ArchLinux, not multi-architecture/platform support. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed on the Engine example. I revisited it and @jedevc is right, but a simple docs search renders this It seems for the discussion there's the case for both in content, when platform is the same and we do want to distinguish architectures, but that we'd like to introduce the feature as multi-platform as it's seen as capturing both. But it leaves the case of harmonizing the remaining docs content. |
||||||
title: Building multi-platform images | ||||||
- sectiontitle: Docker Compose | ||||||
section: | ||||||
- path: /compose/ | ||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
title: Buildx | ||
title: Working with Buildx | ||
description: Working with Docker Buildx | ||
keywords: build, buildx, buildkit | ||
keywords: build, buildx, BuildKit | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. keywords should be lowercased to me (cc @usha-mandya @thaJeztah) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry, this was a suggestion by the IDE. Makes sense to have a standard. Just out of curiosity, does it need to be lowercase, i.e., otherwise something is broken? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good question; I think either way may work, as long as we're consistently using the same. Looks like the search auto-complete uses them case-insensitive, but it may be that they're used different elsewhere; "elsewhere" could be as part of an URL and/or anchor, in which case it's possible that they're treated as "different tags". I guess "lowercase" would be the safest option for these 🤔 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah indeed there is the auto-complete search but I was thinking about search engines and the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, in your previous example all of them are like Docker, BuildKit, etc. Comment: I agree that making them all lowercase allows less room for creating multiple like (docker, Docker, dockerfile, Dockerfile) in terms of metadata, in terms of the presentation (they are visible) they make more sense with case. It would be great to decouple that. As in, write them in suitable case (rendering) and them being always interpretated in a case-insensitive way(metadata). Question for @usha-mandya do we then change all keywords to be lowercase in our docs sets? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @dockertopia Agree on having a standard for keywords. Suggest we use lowercase going forward. We can create a ticket in the backlog to retrospectively update all keywords to be lowercase. It's not a high priority as the search results are case-insensitive. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks @usha-mandya. Sounds good. |
||
redirect_from: | ||
- /buildx/working-with-buildx/ | ||
--- | ||
|
@@ -47,115 +47,6 @@ automatically added to the "docker images" view by default, whereas when using | |
other drivers, the method for outputting an image needs to be selected | ||
with `--output`. | ||
|
||
## Work with builder instances | ||
|
||
By default, Buildx uses the `docker` driver if it is supported, providing a user | ||
experience very similar to the native `docker build`. Note that you must use a | ||
local shared daemon to build your applications. | ||
|
||
Buildx allows you to create new instances of isolated builders. You can use this | ||
to get a scoped environment for your CI builds that does not change the state of | ||
the shared daemon, or for isolating builds for different projects. You can create | ||
a new instance for a set of remote nodes, forming a build farm, and quickly | ||
switch between them. | ||
|
||
You can create new instances using the [`docker buildx create`](../../engine/reference/commandline/buildx_create.md) | ||
command. This creates a new builder instance with a single node based on your | ||
current configuration. | ||
|
||
To use a remote node you can specify the `DOCKER_HOST` or the remote context name | ||
while creating the new builder. After creating a new instance, you can manage its | ||
lifecycle using the [`docker buildx inspect`](../../engine/reference/commandline/buildx_inspect.md), | ||
[`docker buildx stop`](../../engine/reference/commandline/buildx_stop.md), and | ||
[`docker buildx rm`](../../engine/reference/commandline/buildx_rm.md) commands. | ||
To list all available builders, use [`docker buildx ls`](../../engine/reference/commandline/buildx_ls.md). | ||
After creating a new builder you can also append new nodes to it. | ||
|
||
To switch between different builders, use [`docker buildx use <name>`](../../engine/reference/commandline/buildx_use.md). | ||
After running this command, the build commands will automatically use this | ||
builder. | ||
|
||
Docker also features a [`docker context`](../../engine/reference/commandline/context.md) | ||
command that you can use to provide names for remote Docker API endpoints. Buildx | ||
integrates with `docker context` to ensure all the contexts automatically get a | ||
default builder instance. You can also set the context name as the target when | ||
you create a new builder instance or when you add a node to it. | ||
|
||
## Build multi-platform images | ||
|
||
BuildKit is designed to work well for building for multiple platforms and not | ||
only for the architecture and operating system that the user invoking the build | ||
happens to run. | ||
|
||
When you invoke a build, you can set the `--platform` flag to specify the target | ||
platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or | ||
`darwin/amd64`). | ||
|
||
When the current builder instance is backed by the `docker-container` driver, | ||
you can specify multiple platforms together. In this case, it builds a manifest | ||
list which contains images for all specified architectures. When you use this | ||
image in [`docker run`](../../engine/reference/commandline/run.md) or | ||
[`docker service`](../../engine/reference/commandline/service.md), Docker picks | ||
the correct image based on the node's platform. | ||
|
||
You can build multi-platform images using three different strategies that are | ||
supported by Buildx and Dockerfiles: | ||
|
||
1. Using the QEMU emulation support in the kernel | ||
2. Building on multiple native nodes using the same builder instance | ||
3. Using a stage in Dockerfile to cross-compile to different architectures | ||
|
||
QEMU is the easiest way to get started if your node already supports it (for | ||
example. if you are using Docker Desktop). It requires no changes to your | ||
Dockerfile and BuildKit automatically detects the secondary architectures that | ||
are available. When BuildKit needs to run a binary for a different architecture, | ||
it automatically loads it through a binary registered in the `binfmt_misc` | ||
handler. | ||
|
||
For QEMU binaries registered with `binfmt_misc` on the host OS to work | ||
transparently inside containers, they must be statically compiled and registered | ||
with the `fix_binary` flag. This requires a kernel >= 4.8 and | ||
binfmt-support >= 2.1.7. You can check for proper registration by checking if | ||
`F` is among the flags in `/proc/sys/fs/binfmt_misc/qemu-*`. While Docker | ||
Desktop comes preconfigured with `binfmt_misc` support for additional platforms, | ||
for other installations it likely needs to be installed using | ||
[`tonistiigi/binfmt`](https://github.com/tonistiigi/binfmt){:target="_blank" rel="noopener" class="_"} | ||
image. | ||
|
||
```console | ||
$ docker run --privileged --rm tonistiigi/binfmt --install all | ||
``` | ||
|
||
Using multiple native nodes provide better support for more complicated cases | ||
that are not handled by QEMU and generally have better performance. You can | ||
add additional nodes to the builder instance using the `--append` flag. | ||
|
||
Assuming contexts `node-amd64` and `node-arm64` exist in `docker context ls`; | ||
|
||
```console | ||
$ docker buildx create --use --name mybuild node-amd64 | ||
mybuild | ||
$ docker buildx create --append --name mybuild node-arm64 | ||
$ docker buildx build --platform linux/amd64,linux/arm64 . | ||
``` | ||
|
||
Finally, depending on your project, the language that you use may have good | ||
support for cross-compilation. In that case, multi-stage builds in Dockerfiles | ||
can be effectively used to build binaries for the platform specified with | ||
`--platform` using the native architecture of the build node. A list of build | ||
arguments like `BUILDPLATFORM` and `TARGETPLATFORM` is available automatically | ||
inside your Dockerfile and can be leveraged by the processes running as part | ||
of your build. | ||
|
||
```dockerfile | ||
# syntax=docker/dockerfile:1 | ||
FROM --platform=$BUILDPLATFORM golang:alpine AS build | ||
ARG TARGETPLATFORM | ||
ARG BUILDPLATFORM | ||
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log | ||
FROM alpine | ||
COPY --from=build /log /log | ||
``` | ||
|
||
## High-level build options | ||
|
||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,78 @@ | ||||||
--- | ||||||
title: Building multi-platform images | ||||||
description: Different strategies for building multiplatform images | ||||||
crazy-max marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
keywords: build, buildx, BuildKit, multiplatform images | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
--- | ||||||
BuildKit is designed to work well for building for multiple platforms and not | ||||||
only for the architecture and operating system that the user invoking the build | ||||||
happens to run. | ||||||
|
||||||
When you invoke a build, you can set the `--platform` flag to specify the target | ||||||
platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or | ||||||
`darwin/amd64`). | ||||||
|
||||||
When the current builder instance is backed by the `docker-container` driver, | ||||||
you can specify multiple platforms together. In this case, it builds a manifest | ||||||
list which contains images for all specified architectures. When you use this | ||||||
image in [`docker run`](../../engine/reference/commandline/run.md) or | ||||||
[`docker service`](../../engine/reference/commandline/service.md), Docker picks | ||||||
the correct image based on the node's platform. | ||||||
|
||||||
You can build multi-platform images using three different strategies that are | ||||||
supported by Buildx and Dockerfiles: | ||||||
|
||||||
1. Using the QEMU emulation support in the kernel | ||||||
2. Building on multiple native nodes using the same builder instance | ||||||
3. Using a stage in Dockerfile to cross-compile to different architectures | ||||||
|
||||||
QEMU is the easiest way to get started if your node already supports it (for | ||||||
example. if you are using Docker Desktop). It requires no changes to your | ||||||
Dockerfile and BuildKit automatically detects the secondary architectures that | ||||||
are available. When BuildKit needs to run a binary for a different architecture, | ||||||
it automatically loads it through a binary registered in the `binfmt_misc` | ||||||
handler. | ||||||
|
||||||
For QEMU binaries registered with `binfmt_misc` on the host OS to work | ||||||
transparently inside containers, they must be statically compiled and registered | ||||||
with the `fix_binary` flag. This requires a kernel >= 4.8 and | ||||||
binfmt-support >= 2.1.7. You can check for proper registration by checking if | ||||||
`F` is among the flags in `/proc/sys/fs/binfmt_misc/qemu-*`. While Docker | ||||||
Desktop comes preconfigured with `binfmt_misc` support for additional platforms, | ||||||
for other installations it likely needs to be installed using | ||||||
[`tonistiigi/binfmt`](https://github.com/tonistiigi/binfmt){:target="_blank" rel="noopener" class="_"} | ||||||
image. | ||||||
|
||||||
```console | ||||||
$ docker run --privileged --rm tonistiigi/binfmt --install all | ||||||
``` | ||||||
|
||||||
Using multiple native nodes provide better support for more complicated cases | ||||||
that are not handled by QEMU and generally have better performance. You can | ||||||
add additional nodes to the builder instance using the `--append` flag. | ||||||
|
||||||
Assuming contexts `node-amd64` and `node-arm64` exist in `docker context ls`; | ||||||
|
||||||
```console | ||||||
$ docker buildx create --use --name mybuild node-amd64 | ||||||
mybuild | ||||||
$ docker buildx create --append --name mybuild node-arm64 | ||||||
$ docker buildx build --platform linux/amd64,linux/arm64 . | ||||||
``` | ||||||
|
||||||
Finally, depending on your project, the language that you use may have good | ||||||
support for cross-compilation. In that case, multi-stage builds in Dockerfiles | ||||||
can be effectively used to build binaries for the platform specified with | ||||||
`--platform` using the native architecture of the build node. A list of build | ||||||
arguments like `BUILDPLATFORM` and `TARGETPLATFORM` is available automatically | ||||||
inside your Dockerfile and can be leveraged by the processes running as part | ||||||
of your build. | ||||||
|
||||||
```dockerfile | ||||||
# syntax=docker/dockerfile:1 | ||||||
FROM --platform=$BUILDPLATFORM golang:alpine AS build | ||||||
ARG TARGETPLATFORM | ||||||
ARG BUILDPLATFORM | ||||||
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log | ||||||
FROM alpine | ||||||
COPY --from=build /log /log | ||||||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
--- | ||
title: Using multiple builders | ||
description: How to instantiate and work with multiple builders | ||
keywords: build, buildx, BuildKit, builders, build drivers | ||
--- | ||
By default, Buildx uses the `docker` driver if it is supported, providing a user | ||
experience very similar to the native `docker build`. Note that you must use a | ||
local shared daemon to build your applications. | ||
|
||
Buildx allows you to create new instances of isolated builders. You can use this | ||
to get a scoped environment for your CI builds that does not change the state of | ||
the shared daemon, or for isolating builds for different projects. You can create | ||
a new instance for a set of remote nodes, forming a build farm, and quickly | ||
switch between them. | ||
|
||
You can create new instances using the [`docker buildx create`](../../engine/reference/commandline/buildx_create.md) | ||
command. This creates a new builder instance with a single node based on your | ||
current configuration. | ||
|
||
To use a remote node you can specify the `DOCKER_HOST` or the remote context name | ||
while creating the new builder. After creating a new instance, you can manage its | ||
lifecycle using the [`docker buildx inspect`](../../engine/reference/commandline/buildx_inspect.md), | ||
[`docker buildx stop`](../../engine/reference/commandline/buildx_stop.md), and | ||
[`docker buildx rm`](../../engine/reference/commandline/buildx_rm.md) commands. | ||
To list all available builders, use [`docker buildx ls`](../../engine/reference/commandline/buildx_ls.md). | ||
After creating a new builder you can also append new nodes to it. | ||
|
||
To switch between different builders, use [`docker buildx use <name>`](../../engine/reference/commandline/buildx_use.md). | ||
After running this command, the build commands will automatically use this | ||
builder. | ||
|
||
Docker also features a [`docker context`](../../engine/reference/commandline/context.md) | ||
command that you can use to provide names for remote Docker API endpoints. Buildx | ||
integrates with `docker context` to ensure all the contexts automatically get a | ||
default builder instance. You can also set the context name as the target when | ||
you create a new builder instance or when you add a node to it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about the order, looks confusing to me to have the parent page after a subpage.
As I can see with compose it's the first page: https://docs.docker.com/compose/ so we should be consistent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we discussed I wasn't aware you saw this whole page as a "Build Overview", sure let's rename that page and reorder the pages, then, so 1) Buildx Overview, 2) Install Buildx for consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe
Overview
andInstall
is enough as it's already subpages ofBuildx
section. Don't think we need to repeat ourselves?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see what you mean, that there's hierarchy providing context. But when we need to derive it from the folder context and there's multiple in the doc set, I think it's best for any "just" 'overview' or 'install' to be the ones for the main section (so doc set root index or install) for clear disambiguation in a glance.