[Flatcar SDK] nss-3.101 is not found when building stable-3815.2.5

[aw042]

Description

I'm trying to build the latest Flatcar stable release from source using the SDK (without any modifications yet) and I'm encountering an error that nss-3.101 can't be fetched. I checked https://mirror.release.flatcar-linux.net/portage-stable/distfiles/ and only see nss-3.27.1, nss-3.51, and nss-3.51.1. https://mirror.release.flatcar-linux.net/coreos/distfiles/ also does not have nss-3.101, and https://gentoo.osuosl.org/distfiles/39/ does not seem to have any nss release. From the error logs it seems that after failing those three locations it tried to download nss-3.101 from ftp://ftp.mozilla.org/pub/mozilla.org/security/nss/releases/NSS_3_101_RTM/src/nss-3.101.tar.gz and when I use wget I can successfully download the file. ftp and lftp aren't installed within the SDK container, however sftp is. I'm not quite sure why the ftp download fails.

I think the easiest resolution to this problem would be to update the nss version the latest stable release uses, or to include nss-3.101 on https://mirror.release.flatcar-linux.net

Impact

Using flatcar-sdk-all-3815.0.0, I cannot complete the ./build_packages command to build the stable-3815.2.5 release.

Environment and steps to reproduce

1. Set-up:
   I'm on amd64 architecture running Ubuntu 24.04 LTS WSL using the Flatcar SDK and following the documentation from https://www.flatcar.org/docs/latest/reference/developer-guides/sdk-modifying-flatcar/ and https://github.com/flatcar/scripts

2. Task:
   ./build_packages before ./build_image

3. Action(s):
   Commands run:
   git clone https://github.com/flatcar/scripts.git
git checkout stable-3815.2.5
./run_sdk_container -t
./build_packages

4. Error:
   Here is my error log:

sdk@flatcar-sdk-all-3815_0_0_os-stable-3815_2_5 ~/trunk/src/scripts $ ./build_packages
INFO    update_chroot: Setting up portage...
find: ‘/mnt/host/source/config/portage/repos’: No such file or directory
INFO    update_chroot: Setting up crossdev...
INFO    update_chroot: Updating chroot:
INFO    update_chroot:  chroot version: 3815.2.5+2024-07-28-2054
INFO    update_chroot:  Flatcar version: 3815.2.5+2024-07-28-2210
INFO    update_chroot: Updating basic system packages

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 1.34 s.


Total: 0 packages, Size of downloads: 0 KiB

 * Switching native-compiler to x86_64-pc-linux-gnu-13 ...                                                        [ ok ]
INFO    update_chroot: Updating cross x86_64-cros-linux-gnu toolchain
Installing existing binaries

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 1.01 s.


Total: 0 packages, Size of downloads: 0 KiB

 * Switching to x86_64-cros-linux-gnu-2.40 ...                                                                    [ ok ]
 * Switching cross-compiler to x86_64-cros-linux-gnu-13 ...                                                       [ ok ]
INFO    update_chroot: Maybe removing some hard blocks
INFO    update_chroot: No hard blockers to remove
INFO    update_chroot: Updating all SDK packages

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 9.24 s.

[ebuild     U  ] app-misc/ca-certificates-3.101::coreos [3.95::coreos] 74530 KiB
[ebuild     U  ] coreos-base/update_engine-0.4.10-r19::coreos [0.4.10-r18::coreos] USE="cros_host delta_generator -cros-debug -profiling -symlink-usr" 0 KiB

Total: 2 packages (2 upgrades), Size of downloads: 74530 KiB

>>> Verifying ebuild manifests
>>> Emerging (1 of 2) app-misc/ca-certificates-3.101::coreos
>>> Failed to emerge app-misc/ca-certificates-3.101, Log file:
>>>  '/var/log/portage/app-misc:ca-certificates-3.101:20240728-221120.log'
>>> Jobs: 0 of 2 complete, 1 failed                 Load avg: 0.33, 0.21, 0.13
>>> Downloading 'https://mirror.release.flatcar-linux.net/portage-stable/distfiles/nss-3.101.tar.gz'
--2024-07-28 22:11:20--  https://mirror.release.flatcar-linux.net/portage-stable/distfiles/nss-3.101.tar.gz
Resolving mirror.release.flatcar-linux.net... 136.144.58.113
Connecting to mirror.release.flatcar-linux.net|136.144.58.113|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-28 22:11:20 ERROR 404: Not Found.

>>> Downloading 'https://mirror.release.flatcar-linux.net/coreos/distfiles/nss-3.101.tar.gz'
--2024-07-28 22:11:20--  https://mirror.release.flatcar-linux.net/coreos/distfiles/nss-3.101.tar.gz
Resolving mirror.release.flatcar-linux.net... 136.144.58.113
Connecting to mirror.release.flatcar-linux.net|136.144.58.113|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-28 22:11:20 ERROR 404: Not Found.

>>> Downloading 'http://distfiles.gentoo.org/distfiles/39/nss-3.101.tar.gz'
--2024-07-28 22:11:20--  http://distfiles.gentoo.org/distfiles/39/nss-3.101.tar.gz
Resolving distfiles.gentoo.org... 89.187.177.16, 156.146.36.23, 89.187.177.17, ...
Connecting to distfiles.gentoo.org|89.187.177.16|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-28 22:11:21 ERROR 404: Not Found.

>>> Downloading 'ftp://ftp.mozilla.org/pub/mozilla.org/security/nss/releases/NSS_3_101_RTM/src/nss-3.101.tar.gz'
--2024-07-28 22:11:21--  ftp://ftp.mozilla.org/pub/mozilla.org/security/nss/releases/NSS_3_101_RTM/src/nss-3.101.tar.gz
           => ‘/mnt/host/source/.cache/distfiles/nss-3.101.tar.gz.__download__’
Resolving ftp.mozilla.org... 34.117.35.28
Connecting to ftp.mozilla.org|34.117.35.28|:21... failed: Connection refused.
!!! Couldn't download 'nss-3.101.tar.gz'. Aborting.
 * Fetch failed for 'app-misc/ca-certificates-3.101', Log file:
 *  '/var/log/portage/app-misc:ca-certificates-3.101:20240728-221120.log'

 * Messages for package app-misc/ca-certificates-3.101:
 * Log file: /var/log/portage/app-misc:ca-certificates-3.101:20240728-221120.log

 * Fetch failed for 'app-misc/ca-certificates-3.101', Log file:
 *  '/var/log/portage/app-misc:ca-certificates-3.101:20240728-221120.log'
ERROR   update_chroot: script called: update_chroot '--toolchain_boards=amd64-usr' '--usepkg' '--nousepkgonly' '--getbinpkg'
ERROR   update_chroot: Backtrace:  (most recent call is last)
ERROR   update_chroot:   file update_chroot, line 257, called: die_err_trap 'sudo -E ${EMERGE_CMD} "${EMERGE_FLAGS[@]}" coreos-devel/sdk-depends world' '1'
ERROR   update_chroot:
ERROR   update_chroot: Command failed:
ERROR   update_chroot:   Command 'sudo -E ${EMERGE_CMD} "${EMERGE_FLAGS[@]}" coreos-devel/sdk-depends world' exited with nonzero code: 1
ERROR   setup_board: script called: setup_board '--quiet' '--board=amd64-usr' '--regen_configs' '--usepkg' '--nousepkgonly' '--getbinpkg' '--notoolchainpkgonly'
ERROR   setup_board: Backtrace:  (most recent call is last)
ERROR   setup_board:   file setup_board, line 214, called: die_err_trap '"${SRC_ROOT}/scripts"/update_chroot ${UPDATE_ARGS}' '1'
ERROR   setup_board:
ERROR   setup_board: Command failed:
ERROR   setup_board:   Command '"${SRC_ROOT}/scripts"/update_chroot ${UPDATE_ARGS}' exited with nonzero code: 1
ERROR   setup_board:   (Note bash sometimes misreports "command not found" as exit code 1 instead of 127)
ERROR   build_packages: script called: build_packages
ERROR   build_packages: Backtrace:  (most recent call is last)
ERROR   build_packages:   file build_packages, line 121, called: die_err_trap '"${SCRIPTS_DIR}"/setup_board --quiet --board=${FLAGS_board} "${UPDATE_ARGS[@]}"' '1'
ERROR   build_packages:
ERROR   build_packages: Command failed:
ERROR   build_packages:   Command '"${SCRIPTS_DIR}"/setup_board --quiet --board=${FLAGS_board} "${UPDATE_ARGS[@]}"' exited with nonzero code: 1
ERROR   build_packages:   (Note bash sometimes misreports "command not found" as exit code 1 instead of 127)

Expected behavior

./build_packages should succeed without errors so that ./build_image can work.

[jepio]

looking into this. it appears that manifest got out of sync with upstream distfiles, so the mirroring has been failing.

[jepio]

Upstream SRC_URI no longer works, so our mirroring has been failing. Opened a PR to switch to an https url.

I've pushed the missing file at https://mirror.release.flatcar-linux.net/coreos/distfiles/nss-3.101.tar.gz. If you find it still doesn't work, please feel free to re-open the issue.

[aw042]

Upstream SRC_URI no longer works, so our mirroring has been failing. Opened a PR to switch to an https url.

I've pushed the missing file at https://mirror.release.flatcar-linux.net/coreos/distfiles/nss-3.101.tar.gz. If you find it still doesn't work, please feel free to re-open the issue.

I can confirm it works now. I was able to build the image, use ./image_to_vm.sh, and successfully run it in Hyper-V Manager. I appreciate how quickly you resolved this issue. Keep up the great work!

P.S. I'm crossing my fingers that Flatcar is approved as a CNCF incubating project :)