Building ISO using Docker instead of Vagrant

[Nova38]

Is your feature request related to a problem? Please describe.
There are two main problems that this feature request is related to.

1. The major issue comes when trying to develop small changes to the build scripts or files included in the live-build configuration.

   1. When running with the vagrant script it rebuilds the whole image every time and doesn't cache any of the steps. The build process takes a decent amount of time to run and so when it fails at a late point in the build process it will have to go through the process of downloading packages and building the local debs every time.
   2. Debugging small changes to the build configuration scripts is also a quick way to hit the limit on the maxmind api. There was an issue/pull request (allow specifying alternate download location for MaxMind GeoIP database files idaholab/Malcolm#565) that added support to specify a location for a local copy of the files, however it doesn't save the files when it downloads them so. Using docker multi-stage build we can cache the files in a layer and then copy from that into the final build. Using docker could also let us pass in the api key as a docker secret.

    FROM base-deps AS maxmind
    # COPY --link --from=shared '/bin/maxmind-mmdb-download.sh' /maxmind-mmdb-download.sh
    RUN --mount=type=bind,from=shared,source='/bin/maxmind-mmdb-download.sh',target="/maxmind-mmdb-download.sh" \
        --mount=type=secret,id=MAXMIND_GEOIP_DB_LICENSE_KEY,env=MAXMIND_GEOIP_DB_LICENSE_KEY,required=true \
        /maxmind-mmdb-download.sh -k $MAXMIND_GEOIP_DB_LICENSE_KEY
   

   3. Issues running the vagrant build on windows using wsl.
      1. While vagrant does support running in wsl (https://developer.hashicorp.com/vagrant/docs/other/wsl) on windows it has some issues surrounding file ownership as it has to be in the windows filesystem. I have ran into several strange issues, especially when running the build script several times.

Describe the solution you'd like
Add a way to build the iso's using docker instead of vagrant and the raw build.sh script. The docker build would then use multi-stage builds to build out the different subparts of the prep (such as the yara debs that are built or the maxmind api download) in a way that the end products are able to be added to the final image without invalidating the other steps. This will speed up the process as these steps would be able to run concurrently and wouldn't need to be rerun if their are no changes to those parts of the build process. This will make debugging and making small iterations to the build process, such as for a custom build, quicker.

There are also a few other advantages to this. Building a base image with all the dependencies for live-build would be able to be reused with both the hedgehog and the malcom iso's. This would also be able to be slotted into the github action flow to build the images.

Describe alternatives you've considered

There are a few alternatives i have considered that all have some drawbacks.

• The first is to run the build.sh in the wsl instances directly and split the cleanup trap into is own script.
  • If the trap is disabled to allow the build process to be stepped through it causes some trouble when wanting to completely clean the build enverment after stepping through the process. If something gets messed-up in the system or repo that is not being tracked it makes rerunning the build process
• To use the vagrant wsl-windows bridge as shown in the link above.
• It has some file system issues and also slows down the build process as the git repo must be in the windows file system and there are major proffomence penalites
• For quicly being able to create modified iso - for instance only changing out the preseed - you can flash the iso to a usb drive with a tool like rufus in isohybrid mode and then manually edit the files in place.

Additional context

Their is also some repeated steps in the configuration for the malcom and the hedgehog iso build envs. Using Docker could let us make a single base docker image that is then able to be used by both the iso build processes.

This could still reuse ghcr.io/mmguero/qemu-live-iso:latest image as the final stage of the build process and directly copy over the iso from the builid context.

Docker also has support for HereDocs which could simplify the build scripts creation of files and allow inlineing of multi-line scripts without having to write '' at the end.

COPY <<EOF  config/package-lists/firmwares.list.chroot
firmware-linux-free 
firmware-linux-nonfree 
firmware-misc-nonfree 
firmware-amd-graphics 
firmware-iwlwifi 
firmware-atheros 
linux-headers-amd64
EOF

COPY  --link ./interface/ ./config/includes.chroot/opt/sensor/
RUN <<EOF
    set -e
    sed -i "s@/home/sensor/sensor_interface@/opt/sensor@g" ./config/includes.chroot/opt/sensor/kiosk.service
    sed -i "s/CAPTURE_INTERFACE=.*/CAPTURE_INTERFACE=xxxx/g" ./config/includes.chroot/opt/sensor/sensor_ctl/control_vars.conf
EOF

RUN <<EOF
  # configure installation options
  sed -i "s@^\(title-text[[:space:]]*:\).*@\1 \"Hedgehog Linux $IMAGE_VERSION $(date +'%Y-%m-%d %H:%M:%S')\"@g" ./config/bootloaders/grub-pc/live-theme/theme.txt
  cp ./config/includes.binary/install/preseed_multipar.cfg ./config/includes.binary/install/preseed_multipar_crypto.cfg
  cp ./config/includes.binary/install/preseed_base.cfg ./config/includes.binary/install/preseed_minimal.cfg
  sed -i "s@\(partman-auto/method[[:space:]]*string[[:space:]]*\)lvm@\1crypto@g" ./config/includes.binary/install/preseed_multipar_crypto.cfg
  sed -i "s@\(/etc/capture_storage_format\)@\1.crypt@g" ./config/includes.binary/install/preseed_multipar_crypto.cfg
  sed -i "s@\(/etc/capture_storage_format\)@\1.none@g" ./config/includes.binary/install/preseed_minimal.cfg
EOF

[Nova38]

Another thing that is along the same lines is that some of the lb config, lb clean, and lb build can be moved into separate scripts in the '/config/auto/{config,clean,build}' scripts. Here is a link to the docs Live-Build managing-a-configuration.

[Nova38]

Also I am working on some custom builds of hedgehog for a project at LLNL and got permission to contribute some of the work back upstream, so if there is interest in this I could share some of the implementation work i have been doing on it

[mmguero]

Thanks for logging the request! At the moment I'm not sure what kind of development cycles I've got to work on this, but I can see the value of it. Sure, we'd be interested in your implementation work.

Some other links that might be useful might include the GitHub build workflows for hedgehog and Malcolm ISOs, which are not done in vagrant but are done in GitHub ubuntu worker runners.