Perform Image Maintenance #14
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pipenv is used to manage the requirements for the script that serves as the entrypoint for the Docker image. Adding it to the development requirements was an oversight as it is needed to manage the Pipfile files in src/.
Specify the minor Python version we currently build against in the Pipfile. Run `pipenv update` to update the entrypoint script's requirements.
This updates the Python version specified from 3.9 to the more specific 3.9.6 in this project. This will help ensure repeatable builds because the underlying Docker image should not change significantly when a new Python patch version is released.
Update pip, pipenv, setuptools, and wheel versions to their latest in the Dockerfile. Update the libxml2-dev package to its latest version since it is a requirement.
The serverless-chrome approach is fraught with dependency woes. It is built for Amazon Linux 2 usage and running it on a Debian-based image is unwieldy. As a result we simplify things by installing Chromium with apt and using that binary instead.
Chrome is a specific build of Chromium by Google. We are not using this so the argument name should correctly reflect that distinction.
I bumped into this during testing. This way the run should not bomb out at any point in the process of retrieving a hash for a domain.
The libxml2-dev and libxslt1-dev packages are dependencies of the chromium package but are required by the Python packages we are installing. If the chromium dependency were to change it would cause issues in building the Docker image.
Per https://packages.debian.org/buster/chromium there is no `90.0.4430.212-1~deb10u1` release available for the `arm64` platform. As a result we much use the older `89.0.4389.114-1~deb10u1` version for now. I have not been able to determine why that is the only platform out of the four for the package that has an older build. The next time dependencies are updated this will hopefully not be an issue.
Since we have to pin on an older version of the chromium package (from the viewpoint of some of the platforms) we need to pin the chromium-common package it depends on to the same version to ensure installation.
Static values like this should be stored in variables to improve visability and maintainability.
Add information about the two command options for the script.
mcdonnnj
added
improvement
jsf9k
approved these changes
Jul 21, 2021
dav3r
approved these changes
Jul 21, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗣 Description
This PR is a general maintenance update that updates dependencies and does a dash of code cleanup on the entrypoint Python script.
💭 Motivation and context
I need to implement some functionality improvements and at this point the image dependencies are out-of-date and it does not build.
🧪 Testing
I have tested this configuration locally using both
docker runanddocker-compose. Automated tests also pass.✅ Checklist