Building Bottlerocket aws-dev variant using twoliter

[rajivr]

I am new to Bottlerocket, and I am attempting to build aws-dev variant using twoliter.

I am using AL 2023 (al2023-ami-2023.5.20241001.1-kernel-6.1-x86_64), with docker, git, make, rust, cargo and lz4 installed from the yum repositories.

Following are the twoliter and cargo-make versions (installed using GitHub releases)

[ec2-user@ip-10-0-0-25 bottlerocket-1.24.1]$ twoliter --version
twoliter 0.4.7
[ec2-user@ip-10-0-0-25 bottlerocket-1.24.1]$ cargo make --version
cargo-make 0.37.22

When I attempt to build Bottlerocket v1.24.1 using twoliter build variant aws-dev, I am getting the following error.

[ec2-user@ip-10-0-0-25 bottlerocket-1.24.1]$ twoliter build variant aws-dev
[2024-10-12T08:06:54Z WARN  twoliter::project] A Release.toml file was found. Release.toml is deprecated. Please remove it from your project.
[2024-10-12T08:06:54Z INFO  twoliter::project::lock] Resolving project references to check against lock file

[...]

2024-10-12T08:06:58Z INFO  twoliter::project::lock::image] Resolving dependency image dependency '[email protected]/bottlerocket/bottlerocket-sdk:v0.45.0'.
[cargo-make] INFO - cargo make 0.37.22
[cargo-make] INFO -
[cargo-make] INFO - Build File: /home/ec2-user/bottlerocket-1.24.1/build/tools/Makefile.toml
[cargo-make] INFO - Task: build
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: setup
[cargo-make] INFO - Running Task: setup-build
[cargo-make] INFO - Running Task: fetch-sdk
[cargo-make] INFO - Running Task: fetch-sources
error: failed to select a version for `env_logger`.
    ... required by package `settings-extension-oci-defaults v0.1.0 (https://github.com/bottlerocket-os/bottlerocket-settings-sdk?tag=bottlerocket-settings-models-v0.5.0#ae697ef7)`
    ... which satisfies git dependency `settings-extension-oci-defaults` (locked to 0.1.0) of package `bottlerocket-settings-models v0.5.0 (https://github.com/bottlerocket-os/bottlerocket-settings-sdk?tag=bottlerocket-settings-models-v0.5.0#ae697ef7)`
    ... which satisfies git dependency `bottlerocket-settings-models` (locked to 0.5.0) of package `models v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/models)`
    ... which satisfies path dependency `models` (locked to 0.1.0) of package `apiclient v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/api/apiclient)`
    ... which satisfies path dependency `apiclient` (locked to 0.1.0) of package `schnauzer v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/api/schnauzer)`
    ... which satisfies path dependency `schnauzer` (locked to 0.1.0) of package `migration-helpers v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/api/migration/migration-helpers)`
    ... which satisfies path dependency `migration-helpers` (locked to 0.1.0) of package `add-hostname-override-source v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/settings-migrations/v1.21.0/add-hostname-override-source)`
versions that meet the requirements `^0.11` (locked to 0.11.5) are: 0.11.5

the package `settings-extension-oci-defaults` depends on `env_logger`, with features: `anstream` but `env_logger` does not have these features.
 It has an optional dependency with that name, but that dependency uses the "dep:" syntax in the features table, so it does not have an implicit feature with that name.


failed to select a version for `env_logger` which could resolve this conflict
Error while executing command, exit code: 101
Error: Command was unsuccessful, exit code 105

Would you be able to please share your thoughts on what might be happening here and how I might be able to fix it?

Thank you!

[ytsssun]

Hi @rajivr, I assume you have checked out the https://github.com/bottlerocket-os/bottlerocket/blob/develop/BUILDING.md for the building instruction?

Have you tried using simply cargo make as recommended from the BUILDING.md?

cargo make -e BUILDSYS_VARIANT=aws-dev

The error log you provided seems to be caused by a locked version for package env_logger. Have you tried twoliter update to refresh the lock?

[rajivr]

@ytsssun Thanks for the reply.

Using cargo make fails in fetch-sources task. It looks like build/tools/Makefile.toml is updated every time top level cargo make is executed. Therefore I am unable to figure out where exactly cargo fetch might be failing.

Following is the output that I am currently getting.

Could you please let me know if you are you able to reproduce the error on your side?

$ cargo make -e BUILDSYS_VARIANT=aws-dev fetch-sources
[cargo-make] INFO - cargo make 0.37.22
[cargo-make] INFO -
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: fetch-sources
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: install-twoliter
Found Twoliter v0.4.6 installed.
Skipping installation.
[cargo-make] INFO - Execute Command: "/home/ec2-user/bottlerocket-1.24.1/tools/twoliter/twoliter" "--log-level=info" "make" "fetch-sources" "--project-path=/home/ec2-user/bottlerocket-1.24.1/Twoliter.toml" "--cargo-home=/home/ec2-user/bottlerocket-1.24.1/.cargo" "--"
[2024-10-15T08:04:02Z WARN  twoliter::project] A Release.toml file was found. Release.toml is deprecated. Please remove it from your project.
[2024-10-15T08:04:02Z INFO  twoliter::project::lock] Resolving SDK project reference to check against lock file
[2024-10-15T08:04:02Z INFO  twoliter::project::lock::image] Resolving dependency image dependency '[email protected]/bottlerocket/bottlerocket-sdk:v0.45.0'.
[cargo-make][1] INFO -
[cargo-make][1] INFO - Build File: /home/ec2-user/bottlerocket-1.24.1/build/tools/Makefile.toml
[cargo-make][1] INFO - Task: fetch-sources
[cargo-make][1] INFO - Profile: development
[cargo-make][1] INFO - Running Task: setup
[cargo-make][1] INFO - Running Task: fetch-sources
error: failed to select a version for `env_logger`.
    ... required by package `settings-extension-oci-defaults v0.1.0 (https://github.com/bottlerocket-os/bottlerocket-settings-sdk?tag=bottlerocket-settings-models-v0.5.0#ae697ef7)`
    ... which satisfies git dependency `settings-extension-oci-defaults` (locked to 0.1.0) of package `bottlerocket-settings-models v0.5.0 (https://github.com/bottlerocket-os/bottlerocket-settings-sdk?tag=bottlerocket-settings-models-v0.5.0#ae697ef7)`
    ... which satisfies git dependency `bottlerocket-settings-models` (locked to 0.5.0) of package `models v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/models)`
    ... which satisfies path dependency `models` (locked to 0.1.0) of package `apiclient v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/api/apiclient)`
    ... which satisfies path dependency `apiclient` (locked to 0.1.0) of package `schnauzer v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/api/schnauzer)`
    ... which satisfies path dependency `schnauzer` (locked to 0.1.0) of package `migration-helpers v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/api/migration/migration-helpers)`
    ... which satisfies path dependency `migration-helpers` (locked to 0.1.0) of package `add-hostname-override-source v0.1.0 (/home/ec2-user/bottlerocket-1.24.1/sources/settings-migrations/v1.21.0/add-hostname-override-source)`
versions that meet the requirements `^0.11` (locked to 0.11.5) are: 0.11.5

the package `settings-extension-oci-defaults` depends on `env_logger`, with features: `anstream` but `env_logger` does not have these features.
 It has an optional dependency with that name, but that dependency uses the "dep:" syntax in the features table, so it does not have an implicit feature with that name.


failed to select a version for `env_logger` which could resolve this conflict
Error while executing command, exit code: 101
Error: Command was unsuccessful, exit code 105
Error while executing command, exit code: 1

Doing twoliter update in top level bottlerocket-1.24.1 directory does not seem to help. It looks like it does not change the lock file.

[ec2-user@ip-10-0-0-209 bottlerocket-1.24.1]$ cp Twoliter.lock Twoliter.lock.orig

[ec2-user@ip-10-0-0-209 bottlerocket-1.24.1]$ ./tools/twoliter/twoliter update
[2024-10-15T08:06:05Z WARN  twoliter::project] A Release.toml file was found. Release.toml is deprecated. Please remove it from your project.
[2024-10-15T08:06:05Z INFO  twoliter::project::lock] Resolving project references to create lock file
[2024-10-15T08:06:05Z INFO  twoliter::project::lock::image] Resolving dependency image dependency '[email protected]/bottlerocket/bottlerocket-core-kit:v2.8.4'.
public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:167c95c6092f570a125d89a043eb8287f4d35be836e2514398f93a4995e098fa: Pulling from bottlerocket/bottlerocket-core-kit
Digest: sha256:167c95c6092f570a125d89a043eb8287f4d35be836e2514398f93a4995e098fa
Status: Image is up to date for public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:167c95c6092f570a125d89a043eb8287f4d35be836e2514398f93a4995e098fa
public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:167c95c6092f570a125d89a043eb8287f4d35be836e2514398f93a4995e098fa
public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:6b042babaf11e4803d611764d11138e9bee354f37c23bdeeae0083799f4aa63f: Pulling from bottlerocket/bottlerocket-core-kit
Digest: sha256:6b042babaf11e4803d611764d11138e9bee354f37c23bdeeae0083799f4aa63f
Status: Image is up to date for public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:6b042babaf11e4803d611764d11138e9bee354f37c23bdeeae0083799f4aa63f
public.ecr.aws/bottlerocket/bottlerocket-core-kit@sha256:6b042babaf11e4803d611764d11138e9bee354f37c23bdeeae0083799f4aa63f
[2024-10-15T08:06:09Z INFO  twoliter::project::lock::image] Resolving dependency image dependency '[email protected]/bottlerocket/bottlerocket-sdk:v0.45.0'.

[ec2-user@ip-10-0-0-209 bottlerocket-1.24.1]$ diff -u Twoliter.lock.orig Twoliter.lock

[rajivr]

I was installing rust and cargo from AL 2023 yum repositories. Installing from rustup solved this issue, and I am now able to build the image successfully.

[yeazelm]

@rajivr thanks for reporting back! We recommend installing rust via rustup so its good to know that it is working for you!