Build pipeline

[Mischback]

Create an actual build pipeline:

Stages

• Staging
  • CSS assets -> included in Improve build pipeline #126
    • introduce logic for a development build -> included in Improve build pipeline #126
    • provide the actual cache busting setup -> included in Improve build pipeline #126
      • compilation of stylesheet from theme/mischback/_src/style/style.scss into theme/mischback/static/style.css
      • run sphinx build to generate the HTML output (in .build); this step will place the stylesheet in .build/_static/style.css
      • run PostCSS against .build/_static/style.css
        • using the HTML output as input for purgecss
        • autoprefixer
        • cssnano
      • run PostHTML to perform the actual cache busting
      • run a custom Python script to perform the actual cache busting --> Improve build pipeline #126

Dependencies

• Responsive Image Extension doesn't work for an image included in two different documents #120

Reference

• colorizer, which was one of the sidetracks that significantly impacted building this website, has the additional benefit of having implemented a working cache busting in its v2 branch.
• Related: Integrate images into build #61

[Mischback]

Build Pipeline

Stage 0: Development

• the most recent code lives in the development branch
• development happens in dedicated feature branches, which are then used in a pull request against the development branch
• PRs are processed with a dedicated workflow through Github Actions:
  • linting (several different linters for the different languages, e.g. black for Python, stylelint and prettier for SCSS/SASS)
  • building the project includes several stages, including compilation of SCSS/SASS and TS/JS sources and running sphinx to actually create the static HTML output. Management of the actual build process is handled by the project's Makefile.
  • validating the created build artifact runs several HTML validation tools against the (generated) output to ensure semantically correct HTML. It does NOT check, if everything is working!
• Please note: The website's images are not part of the repository. Their existence is not checked! The (generated) build artifact of the building step is not ready for deployment!.
• Focus of this stage and the associated workflow is code quality.
• MISSING / TODO:
  • Tests
    • Unit Tests for Python code, especially the utility scripts; try to provide unit tests for the (custom) sphinx extensions aswell
    • Unit Tests for TS/JS code (how to unit test a codebase with target browser?)
    • Integration Test for (compiled) JS code (at least target major browsers)
  • Validation of (compiled) CSS stylesheet

Stage 1: Staging

• the current release is always the last commit of the main branch
• staging is initiated by performing a PR against main
• PRs are processed with a dedicated workflow in Github Actions:
  • linting as described above; this is only a precaution, result should not deviate from development branch.
  • building: This step is a heavy deviation from Stage 0:
    • obtain image sources (tbd)
    • create responsive images
    • create (minified) CSS stylesheet
    • create (minified) JS
    • run sphinx build process
    • cache busting
    • tidy HTML sources
  • validating: This step is a deviation from Stage 0:
    • (still) perform HTML validation as in Stage 0
      • include checks for image sources!
    • perform CSS validation
    • check for moved documents
      • obtain current live sitemap.xml
      • launch build artifact in a webserver (local/Docker) and try to access documents by their live URI
      • This should provide warnings, but does not FAIL the build
    • ensure no 404
      • launch build artifact in a webserver (local/Docker) and scrape the whole site (obviously don't follow external links; they are covered already by sphinx linkcheck and possibly by the HTML validators).
    • Integration Tests / End-to-End Tests
      • Selenium, Playwright

Stage 2: Release

• a workflow is triggered for a commit to main branch; the commit should be a merge commit from a staging branch
• it is an extension to the workflow described in Stage 1:
  • pick up the artifact from building and apply maximum brotli and gzip compression to everything
  • re-upload as the final artifact

[Mischback]

Resources

• run a HTTP server in a Github Actions workflow
  • https://github.com/marketplace/actions/http-server-action