Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setup_board, update_chroot, dev container: use new bincache mirror #159

Merged
merged 3 commits into from
Oct 7, 2021

Conversation

t-lo
Copy link
Member

@t-lo t-lo commented Oct 5, 2021

This updates the default settings in build scripts to use https://mirror.release.flatcar-linux.net/ instead of the google storage bucket if no binhost or FLATCAR_DEV_BUILDS is specified.

Defaults are updated for

  • update_chroot (runs at SDK initialisation time)
  • setup_board (creates /build/[ARCH]/) chroots
  • the development container
  • set_version

Testing done

  • run ./update_chroot with and without FLATCAR_DEV_BUILDS set, verified that https://mirror.release.flatcar-linux.net/ is used by default, the URL in FLATCAR_DEV_BUILDS otherwise
  • run ./setup_board with and without FLATCAR_DEV_BUILDS set, same as above
  • run ./build_packages and verify https://mirror.release.flatcar-linux.net/ is used as bincache
  • run ./build image prodtar container and verify dev container uses https://mirror.release.flatcar-linux.net/
  • run full CI build (incl. SDK) (http://jenkins.infra.kinvolk.io:8080/job/os/job/manifest/3750/cldsv/)

@t-lo t-lo requested a review from a team October 5, 2021 11:06
set_version Show resolved Hide resolved
setup_board Outdated
@@ -27,8 +27,10 @@ DEFINE_boolean getbinpkg "${FLAGS_TRUE}" \
"Download binary packages from remote repository."
DEFINE_string getbinpkgver "" \
"Use binary packages from a specific version."

get_board_and_variant $FLAGS_board $FLAGS_variant
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FLAGS_board and FLAGS_variant are not yet evaluated at this point, no? It happens below, after the line with # Parse command line flags, I think.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Currently it only outputs the defaults. Since get_board_and_variant is only called to populate BOARD for the succeeding help text I'll just change the help text and remove get_board_and_variant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BOARD variable is used below also for other things, so I don't think get_board_and_variant call should be completely removed. Just called after the flags are evaluated, like it used to be.

setup_board Outdated
@@ -149,7 +149,6 @@ if [[ "${FLAGS_usepkgonly}" -eq "${FLAGS_TRUE}" ]]; then
done
fi

get_board_and_variant $FLAGS_board $FLAGS_variant
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this should stay, I think. Otherwise BOARD, VARIANT and BOARD_VARIANT will not be initialized.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh no I removed it entirely instead of moving it back where it was :-/ Will fix.

This updates the default settings in build scripts to use
https://mirror.release.flatcar-linux.net/
instead of the google storage bucket if no binhost or FLATCAR_DEV_BUILDS
is specified.

Defaults are updated for
* update_chroot (runs at SDK initialisation time)
* setup_board (creates /boards/[ARCH]/) chroots
* the development container
* set_version
@t-lo t-lo force-pushed the t-lo/use-new-bincache-mirror branch from 3649ca5 to 51aac23 Compare October 5, 2021 13:03
@t-lo t-lo requested review from krnowak and a team October 5, 2021 13:32
Comment on lines 8 to 12
if [ "${image_group}" == "developer" ]; then
echo "https://storage.googleapis.com/flatcar-jenkins/${image_group}/boards/${BOARD}/${FLATCAR_VERSION}/${image_path}"
echo "${FLATCAR_DEV_BUILDS}/${image_group}/boards/${BOARD}/${FLATCAR_VERSION}/${image_path}"
else
echo "https://storage.googleapis.com/flatcar-jenkins/boards/${BOARD}/${FLATCAR_VERSION_ID}/${image_path}"
echo "${FLATCAR_DEV_BUILDS}/boards/${BOARD}/${FLATCAR_VERSION_ID}/${image_path}"
fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about it, this won't work in the CI. The CI will set FLATCAR_DEV_BUILDS to a custom value (the private google bucket) so the official mirror will not be used when the dev container is built via CI.
I'll introduce a command line option to build_image to set the dev container's binhost.

@t-lo t-lo requested review from krnowak and a team October 5, 2021 14:39
@t-lo
Copy link
Member Author

t-lo commented Oct 5, 2021

OK, I think we're good to go now.

set_version Outdated Show resolved Hide resolved
@t-lo t-lo requested review from krnowak and pothos October 7, 2021 09:56
@t-lo t-lo merged commit 7f80cb1 into main Oct 7, 2021
@t-lo t-lo deleted the t-lo/use-new-bincache-mirror branch October 7, 2021 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants