Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci/regressions): test respec-w3c differences across branches #3428

Merged
merged 17 commits into from
Apr 13, 2021
Merged
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/workflows/regressions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Regressions
on: workflow_dispatch

env:
PUPPETEER_SKIP_CHROMIUM_DOWNLOAD: 1
PUPPETEER_EXECUTABLE_PATH: /usr/bin/google-chrome
FORCE_COLOR: 1

jobs:
build:
name: Build W3C profile
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: 12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so olde?

Copy link
Member Author

@sidvishnoi sidvishnoi Apr 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using v12 everywhere else and I think spec-generator is on v12 also. So, should move all of them together. Breaking change?
Node v12 goes into EOL 2022-04-30 FWIW.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be tied to LTS instead (everywhere)... 12 seems way too limiting.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- uses: actions/cache@v1
with:
path: ~/.npm
key: ${{ runner.os }}-node-12-${{ hashFiles('**/package-lock.json') }}
- run: npm ci
- run: npm run build:w3c
- uses: actions/upload-artifact@v2
with:
name: builds
path: |
builds/respec-w3c.js*
builds/respec-worker.js*
builds/respec-highlight.js*
retention-days: 3

test:
name: Diff ${{ matrix.source }}
runs-on: ubuntu-latest
needs: build
strategy:
matrix:
source:
- https://w3c.github.io/manifest/
- https://w3c.github.io/payment-request/
- https://w3c.github.io/trace-context/
sidvishnoi marked this conversation as resolved.
Show resolved Hide resolved
fail-fast: false
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: 12
- uses: actions/cache@v1
with:
path: ~/.npm
key: ${{ runner.os }}-node-12-${{ hashFiles('**/package-lock.json') }}
- run: npm ci --production --ignore-scripts
- uses: actions/download-artifact@v2
with:
name: builds
path: ./builds/
- run: git status
- run: node ./tools/respec2html ${{ matrix.source }} before.html -t 30 --verbose
- run: node ./tools/respec2html ${{ matrix.source }} after.html -t 30 --verbose --use-local
- name: Install git-delta # prettier git diff
run: |
set -v
wget -q -O delta.tar.gz https://github.com/dandavison/delta/releases/download/0.7.1/delta-0.7.1-x86_64-unknown-linux-gnu.tar.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wonder if this versioned software will become a problem to maintain? Probably not... just a thought.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of converting it into a separate GitHub action, which I can update as needed.
cargo install took ~4min, so had to use a pre-compiled binary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but time taken on Ci shouldn't be an issue right? That's the "cost of doing business".

Copy link
Member Author

@sidvishnoi sidvishnoi Apr 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues now as I've moved this into a separate GitHub action. It takes 1s to install rather than 4min.

is 4min too much for a CI job? for git diff, yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your meme game is strong, Padawan.

mkdir -p delta
tar xzf delta.tar.gz -C delta/ --strip-components=1
echo "$(pwd)/delta/" >> $GITHUB_PATH
- name: git diff
run: |
set -o pipefail
git diff --exit-code --no-index before.html after.html | delta --diff-so-fancy
exit $?