Provide build-time variables to Dockerfile when building images #2214
Labels
Comments
|
Hey @leovct , this is a great idea! We don't have anyone else currently using build args so we're likely not going to be able to prioritize this soon, but would you be open to submitting a PR? The relevant code lives here: |
This was referenced Mar 5, 2024
Closed
|
Hey! here's some pointers to get you started in the codebase (lmk if I can provide more detail elsewhere) build_args need to get down to ImageBuildOptions.BuildArgs here: This function is where we actually call the docker client to build the image. The function gets called here: LMK and I can provide some more direciton on navigating the types of the codebase (note: we have two representations of ImageBuildSpec (one in our Starlark engine, one in our container engine library that you'll have to add build_args to! Thanks for the contribution! |
github-merge-queue bot
pushed a commit
that referenced
this issue
Mar 12, 2024
## Description:
- [x] Add a new parameter `build_args` in `ImageBuildSpec`.
- [x] Update the logic of the image builder to take into accounts the
build arguments.
- [x] Implement tests (unit + e2e).
- [x] Update docs.
- [x] Test in a real scenario.
### Tests in a real scenario
I updated the `Dockerfile` under
`internal_testsuites/starlark_image-build-package` to expect a build
argument called `BUILD_ARG` and equal to `VALUE`.
```Dockerfile
FROM alpine:latest
WORKDIR /app
COPY . .
# The idea is to test if the build argument has been provided.
# The following commands would not work:
# $ docker build -t image-build-package-test .
# Error: the build argument BUILD_ARG has not been provided or is not equal to VALUE
# $ docker build -t image-build-package-test --build-arg "BUILD_ARG=WRONG_VALUE" .
# Error: the build argument BUILD_ARG has not been provided or is not equal to VALUE
# But this command would work:
# $ docker build -t image-build-package-test --build-arg "BUILD_ARG=VALUE" .
ARG BUILD_ARG="DEFAULT_VALUE"
RUN if [ "$BUILD_ARG" != "VALUE" ]; then \
echo "Error: the build argument BUILD_ARG has not been provided or is not equal to VALUE"; \
exit 1; \
fi
CMD ["echo", "Hello, Kurtosis!"]
```
I then performed three tests to check that the feature was implemented
correctly:
- Test without any build arguments defined.
- Test with the correct build argument defined but with a wrong value.
- Test with the correct build argument defined with the correct value.
For the first two tests, as expected, they fail because the image can't
be built.
```bash
$ ktdebug run --enclave test internal_testsuites/starlark/image-build-package
WARN[2024-03-08T12:48:16+01:00] An error occurred getting the running engine's version; you may be running an out-of-date engine version
WARN[2024-03-08T12:48:16+01:00] We expected the running engine version to match format X.Y.Z, but instead got '386fd4-dirty'; this means that we can't verify the API library and engine versions match so you may encounter runtime errors
INFO[2024-03-08T12:48:16+01:00] Creating a new enclave for Starlark to run inside...
INFO[2024-03-08T12:48:18+01:00] Enclave 'test' created successfully
INFO[2024-03-08T12:48:18+01:00] Executing Starlark package at '/Users/leovct/Documents/opensource/kurtosis/internal_testsuites/starlark/image-build-package' as the passed argument 'internal_testsuites/starlark/image-build-package' looks like a directory
INFO[2024-03-08T12:48:18+01:00] Compressing package 'github.com/kurtosis-tech/kurtosis/internal_testsuites/starlark/image-build-package' at 'internal_testsuites/starlark/image-build-package' for upload
INFO[2024-03-08T12:48:18+01:00] Uploading and executing package 'github.com/kurtosis-tech/kurtosis/internal_testsuites/starlark/image-build-package'
There was an error validating Starlark code
Error while validating final environment of script
Caused by: Failed to build the required image 'kurtosistech/service'.
Caused by: Image build for 'kurtosistech/service' failed with the following output:
{"id":"moby.buildkit.trace","aux":"Cm8KR3NoYTI1NjpkYzExOGIxMmZmMmY2MTE4Y2EwNjY2NjhhNmNjOGQ3ZjQyYmYzZWEwYzg3YWY2YTNjZDExNmI4Y2UzMmNkY2ExGiRbaW50ZXJuYWxdIGxvYWQgcmVtb3RlIGJ1aWxkIGNvbnRleHQ="}
{"id":"moby.buildkit.trace","aux":"Cn0KR3NoYTI1NjpkYzExOGIxMmZmMmY2MTE4Y2EwNjY2NjhhNmNjOGQ3ZjQyYmYzZWEwYzg3YWY2YTNjZDExNmI4Y2UzMmNkY2ExGiRbaW50ZXJuYWxdIGxvYWQgcmVtb3RlIGJ1aWxkIGNvbnRleHQqDAiE9quvBhD/qsaLAw=="}
{"id":"moby.buildkit.trace","aux":"CosBCkdzaGEyNTY6ZGMxMThiMTJmZjJmNjExOGNhMDY2NjY4YTZjYzhkN2Y0MmJmM2VhMGM4N2FmNmEzY2QxMTZiOGNlMzJjZGNhMRokW2ludGVybmFsXSBsb2FkIHJlbW90ZSBidWlsZCBjb250ZXh0KgwIhParrwYQ/6rGiwMyDAiE9quvBhD6gMSPAw=="}
{"id":"moby.buildkit.trace","aux":"CosBCkdzaGEyNTY6ZGMxMThiMTJmZjJmNjExOGNhMDY2NjY4YTZjYzhkN2Y0MmJmM2VhMGM4N2FmNmEzY2QxMTZiOGNlMzJjZGNhMRokW2ludGVybmFsXSBsb2FkIHJlbW90ZSBidWlsZCBjb250ZXh0KgwIhParrwYQyv7JjwMyDAiE9quvBhDF0MuPAw=="}
{"id":"moby.buildkit.trace","aux":"CqMBCkdzaGEyNTY6MTY1ZDBjMTIwNmI3ZDMwMzRkYzllMzE2M2QwNmE5Y2UzYjE4NGI3ZTcxNmU1MzliM2UwZjY0ZTQwZTBkYTBmMxJHc2hhMjU2OmRjMTE4YjEyZmYyZjYxMThjYTA2NjY2OGE2Y2M4ZDdmNDJiZjNlYTBjODdhZjZhM2NkMTE2YjhjZTMyY2RjYTEaD2NvcHkgL2NvbnRleHQgLw=="}
{"id":"moby.buildkit.trace","aux":"CrEBCkdzaGEyNTY6MTY1ZDBjMTIwNmI3ZDMwMzRkYzllMzE2M2QwNmE5Y2UzYjE4NGI3ZTcxNmU1MzliM2UwZjY0ZTQwZTBkYTBmMxJHc2hhMjU2OmRjMTE4YjEyZmYyZjYxMThjYTA2NjY2OGE2Y2M4ZDdmNDJiZjNlYTBjODdhZjZhM2NkMTE2YjhjZTMyY2RjYTEaD2NvcHkgL2NvbnRleHQgLyoMCIT2q68GEJXyjJED"}
{"id":"moby.buildkit.trace","aux":"Cr8BCkdzaGEyNTY6MTY1ZDBjMTIwNmI3ZDMwMzRkYzllMzE2M2QwNmE5Y2UzYjE4NGI3ZTcxNmU1MzliM2UwZjY0ZTQwZTBkYTBmMxJHc2hhMjU2OmRjMTE4YjEyZmYyZjYxMThjYTA2NjY2OGE2Y2M4ZDdmNDJiZjNlYTBjODdhZjZhM2NkMTE2YjhjZTMyY2RjYTEaD2NvcHkgL2NvbnRleHQgLyoMCIT2q68GEJXyjJEDMgwIhParrwYQ6JOHlAM="}
{"id":"moby.buildkit.trace","aux":"CpUBCkdzaGEyNTY6MTM1NDljNThhNzZiY2I1ZGFjOWQ1MmJjMzY4YThmYjZiNWNmNzY1OWY5NGUzZmE2Mjk0OTE3Yjg1NTQ2OTc4ZBo8W2ludGVybmFsXSBsb2FkIG1ldGFkYXRhIGZvciBkb2NrZXIuaW8vbGlicmFyeS9hbHBpbmU6bGF0ZXN0KgwIhParrwYQ25TglAM="}
{"id":"moby.buildkit.trace","aux":"CqMBCkdzaGEyNTY6MTM1NDljNThhNzZiY2I1ZGFjOWQ1MmJjMzY4YThmYjZiNWNmNzY1OWY5NGUzZmE2Mjk0OTE3Yjg1NTQ2OTc4ZBo8W2ludGVybmFsXSBsb2FkIG1ldGFkYXRhIGZvciBkb2NrZXIuaW8vbGlicmFyeS9hbHBpbmU6bGF0ZXN0KgwIhParrwYQ25TglAMyDAiF9quvBhCl+4CKAQ=="}
{"id":"moby.buildkit.trace","aux":"CrACCkdzaGEyNTY6MWFhZWYxZjViNTc0YzNhNmU3MzdhODFjZTMxZmI1YzRmOGNmYWZlNTg5ZDkwNWMwZWQ2YjYzMmZmYzMwNzM5NhJHc2hhMjU2OjE0MjBiNTM5NzFkODkzOTE2YzRlMGU4ZDVlZDRlMjRlNjlkMmE0ODFlNzRiNDcxM2M0NzBhYzczNDU1NDU4YTIamwFbNC80XSBSVU4gaWYgWyAiREVGQVVMVF9WQUxVRSIgIT0gIlZBTFVFIiBdOyB0aGVuICAgZWNobyAiRXJyb3I6IHRoZSBidWlsZCBhcmd1bWVudCBCVUlMRF9BUkcgaGFzIG5vdCBiZWVuIHByb3ZpZGVkIG9yIGlzIG5vdCBlcXVhbCB0byBWQUxVRSI7ICAgZXhpdCAxOyBmaQrrAQpHc2hhMjU2OjE0MjBiNTM5NzFkODkzOTE2YzRlMGU4ZDVlZDRlMjRlNjlkMmE0ODFlNzRiNDcxM2M0NzBhYzczNDU1NDU4YTISR3NoYTI1Njo0M2Q4OWU2YzU4NGQyM2U1YjM1ODU2MzQ5ODQxZDBmN2M1NmRlMTA3NDQ3YjNhZjdiY2I4ODEwNjBlMGM1ZjNlEkdzaGEyNTY6MTY1ZDBjMTIwNmI3ZDMwMzRkYzllMzE2M2QwNmE5Y2UzYjE4NGI3ZTcxNmU1MzliM2UwZjY0ZTQwZTBkYTBmMxoOWzMvNF0gQ09QWSAuIC4KpgEKR3NoYTI1Njo0M2Q4OWU2YzU4NGQyM2U1YjM1ODU2MzQ5ODQxZDBmN2M1NmRlMTA3NDQ3YjNhZjdiY2I4ODEwNjBlMGM1ZjNlEkdzaGEyNTY6OWVmN2JlYWVhMDQ3MzA5MTg1MDQ3NzY0ZTNhYTBkNGYyOTk3OWRhY2M4OGU1NzNkYWE5ZDBmODIwNTBiNGFiMRoSWzIvNF0gV09SS0RJUiAvYXBwCr0BCkdzaGEyNTY6OWVmN2JlYWVhMDQ3MzA5MTg1MDQ3NzY0ZTNhYTBkNGYyOTk3OWRhY2M4OGU1NzNkYWE5ZDBmODIwNTBiNGFiMRpyWzEvNF0gRlJPTSBkb2NrZXIuaW8vbGlicmFyeS9hbHBpbmU6bGF0ZXN0QHNoYTI1NjpjNWIxMjYxZDZkM2U0MzA3MTYyNjkzMWZjMDA0ZjcwMTQ5YmFlYmEyYzhlYzY3MmJkNGYyNzc2MWY4ZTFhZDZiCtkBCkdzaGEyNTY6OWVmN2JlYWVhMDQ3MzA5MTg1MDQ3NzY0ZTNhYTBkNGYyOTk3OWRhY2M4OGU1NzNkYWE5ZDBmODIwNTBiNGFiMRpyWzEvNF0gRlJPTSBkb2NrZXIuaW8vbGlicmFyeS9hbHBpbmU6bGF0ZXN0QHNoYTI1NjpjNWIxMjYxZDZkM2U0MzA3MTYyNjkzMWZjMDA0ZjcwMTQ5YmFlYmEyYzhlYzY3MmJkNGYyNzc2MWY4ZTFhZDZiKgwIhfarrwYQ0v3JigEyDAiF9quvBhDz1teKAQ=="}
{"id":"moby.buildkit.trace","aux":"CsQBCkdzaGEyNTY6NDNkODllNmM1ODRkMjNlNWIzNTg1NjM0OTg0MWQwZjdjNTZkZTEwNzQ0N2IzYWY3YmNiODgxMDYwZTBjNWYzZRJHc2hhMjU2OjllZjdiZWFlYTA0NzMwOTE4NTA0Nzc2NGUzYWEwZDRmMjk5NzlkYWNjODhlNTczZGFhOWQwZjgyMDUwYjRhYjEaElsyLzRdIFdPUktESVIgL2FwcCABKgwIhfarrwYQ7PyxiwEyDAiF9quvBhDAsLiLAQ=="}
{"id":"moby.buildkit.trace","aux":"CvkBCkdzaGEyNTY6MTQyMGI1Mzk3MWQ4OTM5MTZjNGUwZThkNWVkNGUyNGU2OWQyYTQ4MWU3NGI0NzEzYzQ3MGFjNzM0NTU0NThhMhJHc2hhMjU2OjQzZDg5ZTZjNTg0ZDIzZTViMzU4NTYzNDk4NDFkMGY3YzU2ZGUxMDc0NDdiM2FmN2JjYjg4MTA2MGUwYzVmM2USR3NoYTI1NjoxNjVkMGMxMjA2YjdkMzAzNGRjOWUzMTYzZDA2YTljZTNiMTg0YjdlNzE2ZTUzOWIzZTBmNjRlNDBlMGRhMGYzGg5bMy80XSBDT1BZIC4gLioMCIX2q68GEN6wvosB"}
{"id":"moby.buildkit.trace","aux":"CocCCkdzaGEyNTY6MTQyMGI1Mzk3MWQ4OTM5MTZjNGUwZThkNWVkNGUyNGU2OWQyYTQ4MWU3NGI0NzEzYzQ3MGFjNzM0NTU0NThhMhJHc2hhMjU2OjQzZDg5ZTZjNTg0ZDIzZTViMzU4NTYzNDk4NDFkMGY3YzU2ZGUxMDc0NDdiM2FmN2JjYjg4MTA2MGUwYzVmM2USR3NoYTI1NjoxNjVkMGMxMjA2YjdkMzAzNGRjOWUzMTYzZDA2YTljZTNiMTg0YjdlNzE2ZTUzOWIzZTBmNjRlNDBlMGRhMGYzGg5bMy80XSBDT1BZIC4gLioMCIX2q68GEN6wvosBMgwIhfarrwYQyOfrlAE="}
{"id":"moby.buildkit.trace","aux":"Cr4CCkdzaGEyNTY6MWFhZWYxZjViNTc0YzNhNmU3MzdhODFjZTMxZmI1YzRmOGNmYWZlNTg5ZDkwNWMwZWQ2YjYzMmZmYzMwNzM5NhJHc2hhMjU2OjE0MjBiNTM5NzFkODkzOTE2YzRlMGU4ZDVlZDRlMjRlNjlkMmE0ODFlNzRiNDcxM2M0NzBhYzczNDU1NDU4YTIamwFbNC80XSBSVU4gaWYgWyAiREVGQVVMVF9WQUxVRSIgIT0gIlZBTFVFIiBdOyB0aGVuICAgZWNobyAiRXJyb3I6IHRoZSBidWlsZCBhcmd1bWVudCBCVUlMRF9BUkcgaGFzIG5vdCBiZWVuIHByb3ZpZGVkIG9yIGlzIG5vdCBlcXVhbCB0byBWQUxVRSI7ICAgZXhpdCAxOyBmaSoMCIX2q68GEPmv7JUB"}
{"id":"moby.buildkit.trace","aux":"Gq4BCkdzaGEyNTY6MWFhZWYxZjViNTc0YzNhNmU3MzdhODFjZTMxZmI1YzRmOGNmYWZlNTg5ZDkwNWMwZWQ2YjYzMmZmYzMwNzM5NhIMCIX2q68GEI710M8BGAEiU0Vycm9yOiB0aGUgYnVpbGQgYXJndW1lbnQgQlVJTERfQVJHIGhhcyBub3QgYmVlbiBwcm92aWRlZCBvciBpcyBub3QgZXF1YWwgdG8gVkFMVUUK"}
{"id":"moby.buildkit.trace","aux":"CqQECkdzaGEyNTY6MWFhZWYxZjViNTc0YzNhNmU3MzdhODFjZTMxZmI1YzRmOGNmYWZlNTg5ZDkwNWMwZWQ2YjYzMmZmYzMwNzM5NhJHc2hhMjU2OjE0MjBiNTM5NzFkODkzOTE2YzRlMGU4ZDVlZDRlMjRlNjlkMmE0ODFlNzRiNDcxM2M0NzBhYzczNDU1NDU4YTIamwFbNC80XSBSVU4gaWYgWyAiREVGQVVMVF9WQUxVRSIgIT0gIlZBTFVFIiBdOyB0aGVuICAgZWNobyAiRXJyb3I6IHRoZSBidWlsZCBhcmd1bWVudCBCVUlMRF9BUkcgaGFzIG5vdCBiZWVuIHByb3ZpZGVkIG9yIGlzIG5vdCBlcXVhbCB0byBWQUxVRSI7ICAgZXhpdCAxOyBmaSoMCIX2q68GEPmv7JUBMgwIhfarrwYQn6i91AE61QFwcm9jZXNzICIvYmluL3NoIC1jIGlmIFsgXCIkQlVJTERfQVJHXCIgIT0gXCJWQUxVRVwiIF07IHRoZW4gICBlY2hvIFwiRXJyb3I6IHRoZSBidWlsZCBhcmd1bWVudCBCVUlMRF9BUkcgaGFzIG5vdCBiZWVuIHByb3ZpZGVkIG9yIGlzIG5vdCBlcXVhbCB0byBWQUxVRVwiOyAgIGV4aXQgMTsgZmkiIGRpZCBub3QgY29tcGxldGUgc3VjY2Vzc2Z1bGx5OiBleGl0IGNvZGU6IDE="}
{"errorDetail":{"message":"process \"/bin/sh -c if [ \\\"$BUILD_ARG\\\" != \\\"VALUE\\\" ]; then echo \\\"Error: the build argument BUILD_ARG has not been provided or is not equal to VALUE\\\"; exit 1; fi\" did not complete successfully: exit code: 1"},"error":"process \"/bin/sh -c if [ \\\"$BUILD_ARG\\\" != \\\"VALUE\\\" ]; then echo \\\"Error: the build argument BUILD_ARG has not been provided or is not equal to VALUE\\\"; exit 1; fi\" did not complete successfully: exit code: 1"}
--- at /Users/leovct/Documents/opensource/kurtosis/container-engine-lib/lib/backend_impls/docker/docker_manager/docker_manager.go:1479 (DockerManager.BuildImage) ---
Container images used in this run:
> kurtosistech/service - locally built
Error encountered running Starlark code.
Made with Kurtosis - https://kurtosis.com
INFO[2024-03-08T12:48:21+01:00] =============================================
INFO[2024-03-08T12:48:21+01:00] || Created enclave: test ||
INFO[2024-03-08T12:48:21+01:00] =============================================
Name: test
UUID: fc14d605d775
Status: RUNNING
Creation Time: Fri, 08 Mar 2024 12:48:16 CET
Flags:
========================================= Files Artifacts =========================================
UUID Name
========================================== User Services ==========================================
UUID Name Ports Status
```
For the last test, it succeeds, meaning the feature has been correctly
implemented! 🥳
```bash
$ ktdebug run --enclave test internal_testsuites/starlark/image-build-package
WARN[2024-03-08T12:49:39+01:00] An error occurred getting the running engine's version; you may be running an out-of-date engine version
WARN[2024-03-08T12:49:39+01:00] We expected the running engine version to match format X.Y.Z, but instead got '386fd4-dirty'; this means that we can't verify the API library and engine versions match so you may encounter runtime errors
INFO[2024-03-08T12:49:39+01:00] Executing Starlark package at '/Users/leovct/Documents/opensource/kurtosis/internal_testsuites/starlark/image-build-package' as the passed argument 'internal_testsuites/starlark/image-build-package' looks like a directory
INFO[2024-03-08T12:49:39+01:00] Compressing package 'github.com/kurtosis-tech/kurtosis/internal_testsuites/starlark/image-build-package' at 'internal_testsuites/starlark/image-build-package' for upload
INFO[2024-03-08T12:49:39+01:00] Uploading and executing package 'github.com/kurtosis-tech/kurtosis/internal_testsuites/starlark/image-build-package'
Container images used in this run:
> kurtosistech/service - locally built
WARNING: Container images with different architecture than expected(amd64):
> kurtosistech/service - arm64
Adding service with name 'service' and image 'kurtosistech/service'
Service 'service' added with service UUID '30bd3754ec864729bf11e2ed9154b4f6'
Starlark code successfully run. No output was returned.
Made with Kurtosis - https://kurtosis.com
Name: test
UUID: fc14d605d775
Status: RUNNING
Creation Time: Fri, 08 Mar 2024 12:48:16 CET
Flags:
========================================= Files Artifacts =========================================
UUID Name
========================================== User Services ==========================================
UUID Name Ports Status
30bd3754ec86 service <none> STOPPED
```
## Is this change user facing?
YES
<!-- If yes, please add the "user facing" label to the PR -->
<!-- If yes, don't forget to include docs changes where relevant -->
## References (if applicable):
<!-- Add relevant Github Issues, Discord threads, or other helpful
information. -->
Resolves #2214
---------
Co-authored-by: Tedi Mitiku <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Background & motivation
I would like to be able to pass build args (or build-time variables) to Dockerfile when building the images with
ImageBuildSpec. This is something I can get around at the moment by executing my instructions at container startup and using environment variables, but it would be nice to have it. It could speed up the startup of devnets considerably.$ docker build --help | grep build-arg --build-arg stringArray Set build-time variablesDesired behaviour
How important is this to you?
Nice to have; this feature would make using Kurtosis more enjoyable.
What area of the product does this pertain to?
Other: anything not covered by the above
The text was updated successfully, but these errors were encountered: