Skip to content

Commit

Permalink
Update prebuild workflows
Browse files Browse the repository at this point in the history
- Use [email protected] for pre-Node.js v10 support
- Harfbuzz is now a separate include
- libgsf-1 is possibly needed for SVG support?
- Embed Windows bundle script in prebuild.yaml
- Update canvas-prebuilt docker image version
- Make text test assertion more flexible. This broke when the docker
  image changed.
- Temporarily rebuild libjpeg in GNU standard location
- Set runpath for copied SOs on Linux
- Update from numworks/setup-msys2@v1 to msys2/setup-msys2@v2
- Update from actions/[email protected] to v2
- Use the prebuild/ files from the commit that was used to trigger the
  prebuild, not the commit of canvas that is being built. See further
  comments in prebuild.yaml.
  • Loading branch information
zbjornson committed Feb 27, 2021
1 parent e476656 commit f0974bc
Show file tree
Hide file tree
Showing 6 changed files with 112 additions and 62 deletions.
131 changes: 97 additions & 34 deletions .github/workflows/prebuild.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,28 @@
# Triggering prebuilds:
# 1. Create a draft release manually using the GitHub UI.
# 2. Set the `jobs.*.strategy.matrix.node` arrays to the set of Node.js versions
# Release procedure:
# 1. Update the changelog and stage it in git.
# 2. Run `yarn version --[major|minor|patch]`. (This tags the commit.)
# 3. Push both the commit and tags to GitHub (`git push --tags Automattic` and
# `git push Automattic HEAD:master`, where "Automattic" is the remote name
# for [email protected]:Automattic/node-canvas.git).
# 4. Create a draft release manually using the GitHub UI for the new tag.
# 5. Set the `jobs.*.strategy.matrix.node` arrays to the set of Node.js versions
# to build for.
# 3. Set the `jobs.*.strategy.matrix.canvas_tag` arrays to the set of Canvas
# 6. Set the `jobs.*.strategy.matrix.canvas_tag` arrays to the set of Canvas
# tags to build. (Usually this is a single tag, but can be an array when a
# new version of Node.js is released and older versions of Canvas need to be
# built.)
# 4. Commit and push this file to master.
# 5. In the Actions tab, navigate to the "Make Prebuilds" workflow and click
# "Run workflow".
# 6. Once the builds succeed, promote the draft release to a full release.
# 7. Commit this file and push to the `prebuilds` branch.
# 8. In the Actions tab, navigate to the "Make Prebuilds" workflow and click
# "Run workflow". Select the `prebuilds` branch so that that branch's
# prebuild.yaml file is used.
# 9. Once the builds succeed, promote the draft release to a full release and
# run npm publish.

# Note that the files in the prebuild/ directory will be used from the commit
# that was used to trigger the prebuild workflow. They will not be from the
# commit/tag that you are building. Because of differences in the provided
# versions of git, this is achieved in a different way on Linux than on Mac and
# Win.

name: Make Prebuilds
on: workflow_dispatch
Expand All @@ -20,30 +33,54 @@ on: workflow_dispatch
# env:
# UPLOAD_TO: "v0.0.1"

# If/when node-canvas moves to Node.js 10+, the latest node-gyp can be used.

jobs:
Linux:
strategy:
matrix:
node: [8, 9, 10, 11, 12, 13, 14]
canvas_tag: [] # e.g. "v2.6.1"
node: [15]
canvas_tag: ["v2.6.1"] # e.g. "v2.6.1"
name: ${{ matrix.canvas_tag}}, Node.js ${{ matrix.node }}, Linux
runs-on: ubuntu-latest
container:
image: chearon/canvas-prebuilt:7
image: chearon/canvas-prebuilt:8
env:
CANVAS_VERSION_TO_BUILD: ${{ matrix.canvas_tag }}
steps:
- uses: actions/checkout@v2
with:
ref: ${{ matrix.canvas_tag }}

# Use the files in the prebuild/ directory from the commit that was used
# to trigger the prebuild workflow. The version of git that's installed on
# this Linux here doesn't support the method used on Mac and Win, so we
# checkout the prebuild branch and copy the files from the prebuild/
# directory in the Build step.
- uses: actions/checkout@v2
with:
ref: ${{ env.GITHUB_SHA }}
path: prebuild-branch

- uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node }}

- name: Reinstall libjpeg in /usr/local
run: |
set -ex
cd /root/libjpeg-*
cd b
cmake -G"Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local ..
make
make install
- name: Build
run: |
npm install -g node-gyp
set -ex
mkdir prebuild
cp -rfv ./prebuild-branch/prebuild/* ./prebuild/
npm install -g [email protected]
npm install --ignore-scripts
. prebuild/Linux/preinstall.sh
cp prebuild/Linux/binding.gyp binding.gyp
Expand All @@ -52,20 +89,25 @@ jobs:
- name: Test binary
run: |
set -ex
cd /root/harfbuzz-* && make uninstall
cd /root/cairo-* && make uninstall
cd /root/pango-* && make uninstall
cd /root/pango-* && cd _build && ninja uninstall
cd /root/libpng-* && make uninstall
cd /root/libjpeg-* && make uninstall
cd /root/libjpeg-* && cd b && make uninstall
cd /root/giflib-* && make uninstall
cd $GITHUB_WORKSPACE && npm test
cd $GITHUB_WORKSPACE
ls build/Release
ldd build/Release/canvas.node
npx mocha test/*.test.js
- name: Make bundle
id: make_bundle
run: . prebuild/tarball.sh

- name: Upload
uses: actions/github-script@0.9.0
uses: actions/github-script@v2
with:
script: |
const fs = require("fs");
Expand Down Expand Up @@ -97,8 +139,8 @@ jobs:
macOS:
strategy:
matrix:
node: [8, 9, 10, 11, 12, 13, 14]
canvas_tag: [] # e.g. "v2.6.1"
node: [15]
canvas_tag: ["v2.6.1"] # e.g. "v2.6.1"
name: ${{ matrix.canvas_tag}}, Node.js ${{ matrix.node }}, macOS
runs-on: macos-latest
env:
Expand All @@ -114,7 +156,9 @@ jobs:

- name: Build
run: |
npm install -g node-gyp
git checkout ${{ matrix.canvas_tag }}
git checkout $GITHUB_SHA -- prebuild/
npm install -g [email protected]
npm install --ignore-scripts
. prebuild/macOS/preinstall.sh
cp prebuild/macOS/binding.gyp binding.gyp
Expand All @@ -123,15 +167,15 @@ jobs:
- name: Test binary
run: |
brew uninstall --force cairo pango librsvg giflib harfbuzz
brew uninstall --force --ignore-dependencies cairo pango librsvg giflib harfbuzz
npm test
- name: Make bundle
id: make_bundle
run: . prebuild/tarball.sh

- name: Upload
uses: actions/github-script@0.9.0
uses: actions/github-script@v2
with:
script: |
const fs = require("fs");
Expand Down Expand Up @@ -163,15 +207,18 @@ jobs:
Win:
strategy:
matrix:
node: [8, 9, 10, 11, 12, 13, 14]
canvas_tag: [] # e.g. "v2.6.1"
node: [15]
canvas_tag: ["v2.6.1"] # e.g. "v2.6.1"
name: ${{ matrix.canvas_tag}}, Node.js ${{ matrix.node }}, Windows
runs-on: windows-latest
env:
CANVAS_VERSION_TO_BUILD: ${{ matrix.canvas_tag }}
steps:
# TODO drop when https://github.com/actions/virtual-environments/pull/632 lands
- uses: numworks/setup-msys2@v1
# GitHub runners now have msys2 installed, but msys is not on the path and
# is apparently slow to start.
# https://github.com/msys2/setup-msys2#setup-msys2
# https://github.com/actions/virtual-environments/pull/632
- uses: msys2/setup-msys2@v2
with:
update: true
path-type: inherit
Expand All @@ -186,15 +233,31 @@ jobs:

- name: Build
run: |
npm install -g node-gyp
git checkout ${{ matrix.canvas_tag }}
git checkout $env:GITHUB_SHA -- prebuild/
npm install -g [email protected]
npm install --ignore-scripts
msys2do . prebuild/Windows/preinstall.sh
msys2do cp prebuild/Windows/binding.gyp binding.gyp
msys2do node-gyp configure
msys2do node-gyp rebuild -j 2
msys2 -c ". prebuild/Windows/preinstall.sh"
msys2 -c "cp prebuild/Windows/binding.gyp binding.gyp"
msys2 -c "node-gyp rebuild -j 2"
- name: Install Depends
run: |
Invoke-WebRequest "http://www.dependencywalker.com/depends22_x64.zip" -OutFile depends22_x64.zip
7z e depends22_x64.zip
- name: Bundle pt 2
shell: msys2 {0}
run: |
./depends.exe -c -oc depends.csv build\\Release\\canvas.node || true
[ -f depends.csv ] || { echo "error invoking depends.exe"; exit 1; }
copies=$(comm -12 \
<(cat depends.csv | cut -d ',' -f2 | sed 's/"//g' | tr '[:upper:]' '[:lower:]' | sort) \
<(find /mingw64/bin -name '*.dll' -printf "%f\n" | tr '[:upper:]' '[:lower:]' | sort) \
- name: Bundle
run: msys2do . prebuild/Windows/bundle.sh

This comment has been minimized.

Copy link
@chearon

chearon Feb 27, 2021

Collaborator

Does it need to be inlined? I'd rather keep as much in those scripts as possible. It makes local testing easy.

This comment has been minimized.

Copy link
@zbjornson

zbjornson Feb 27, 2021

Author Collaborator

I'll un-inline it. Until very recently the files in prebuild/ came from the Canvas git tag that was being built, not the prebuild branch, so modifying prebuild/ files was impossible when trying to fix builds for old versions of Canvas.

Now there's the git checkout $env:GITHUB_SHA -- prebuild/ line on Win and Mac and the clunky second clone on Linux.

for dll in $copies; do
cp /mingw64/bin/$dll build/Release
done;
- name: Test binary
# By not running in msys2, this doesn't have access to the msys2 libs
Expand All @@ -203,10 +266,10 @@ jobs:
- name: Make asset
id: make_bundle
# I can't figure out why this isn't an env var already. It shows up with `env`.
run: msys2do UPLOAD_TO=${{ env.UPLOAD_TO }} CANVAS_VERSION_TO_BUILD=${{ env.CANVAS_VERSION_TO_BUILD}} . prebuild/tarball.sh
run: msys2 -c "UPLOAD_TO=${{ env.UPLOAD_TO }} CANVAS_VERSION_TO_BUILD=${{ env.CANVAS_VERSION_TO_BUILD}} . prebuild/tarball.sh"

- name: Upload
uses: actions/github-script@0.9.0
uses: actions/github-script@v2
with:
script: |
const fs = require("fs");
Expand Down
6 changes: 6 additions & 0 deletions prebuild/Linux/bundle.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
set -ex

copies=$(lddtree.sh -l build/Release/canvas.node | sed -r -e '/^\/lib/d' -e '/canvas.node$/d');

apt install -y patchelf

for so in $copies; do
cp $so build/Release
# Set the run_path for all dependencies.
patchelf --set-rpath '$ORIGIN' build/Release/$(basename $so)

This comment has been minimized.

Copy link
@chearon

chearon Feb 27, 2021

Collaborator

Why does the binary need patching now, but didn't before?

This comment has been minimized.

Copy link
@zbjornson

zbjornson Feb 27, 2021

Author Collaborator

I don't really understand what changed, to be honest. The new tarballs have a slightly different set of library files than the old ones. I'm guessing the new dependency versions added some new function calls across dependencies. In retrospect, I think we've had some cryptic GitHub issues that were resolved by installing libraries that should have been bundled, so we should maybe have been doing this all along.

image

(I haven't manually tested SVG parsing to see if it still works as expected without libcroco...)

This comment has been minimized.

Copy link
@chearon

chearon Feb 27, 2021

Collaborator

We already compile with -rpath $ORIGIN in the custom binding.gyp. That goes to the linker at runtime and, I thought, is used for dependencies-of-dependencies. Using strace seems to verify this. (lddtree output is deceiving, it can't know what the RPATH will be for the libraries which load other libraries). Did the require('canvas') fail without these lines?

done;
4 changes: 3 additions & 1 deletion prebuild/Windows/binding.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
'include_dirs': [
'<!(node -e "require(\'nan\')")',
'D:/a/_temp/msys/msys64/mingw64/include',
'D:/a/_temp/msys/msys64/mingw64/include/harfbuzz',
'D:/a/_temp/msys/msys64/mingw64/include/pango-1.0',
'D:/a/_temp/msys/msys64/mingw64/include/cairo',
'D:/a/_temp/msys/msys64/mingw64/include/libpng16',
Expand All @@ -52,7 +53,8 @@
'D:/a/_temp/msys/msys64/mingw64/include/freetype2',
'D:/a/_temp/msys/msys64/mingw64/include/fontconfig',
'D:/a/_temp/msys/msys64/mingw64/include/librsvg-2.0',
'D:/a/_temp/msys/msys64/mingw64/include/gdk-pixbuf-2.0'
'D:/a/_temp/msys/msys64/mingw64/include/gdk-pixbuf-2.0',
'D:/a/_temp/msys/msys64/mingw64/include/libgsf-1' # NEW
],
'configurations': {
'Debug': {
Expand Down
23 changes: 0 additions & 23 deletions prebuild/Windows/bundle.sh

This file was deleted.

5 changes: 3 additions & 2 deletions prebuild/Windows/preinstall.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# expects node, VS, and MSYS environments to be set up already. does everything else.

deps="cairo-2 png16-16 jpeg-8 pango-1.0-0 pangocairo-1.0-0 gobject-2.0-0 glib-2.0-0 turbojpeg gif-7 freetype-6 rsvg-2-2";
deps="cairo-2 png16-16 jpeg-8 pango-1.0-0 pangocairo-1.0-0 gobject-2.0-0 glib-2.0-0 turbojpeg gif-7 freetype-6 rsvg-2-2 gsf-1-114";

# install cairo and tools to create .lib

Expand All @@ -16,7 +16,8 @@ pacman --noconfirm -S \
mingw64/mingw-w64-x86_64-freetype \
mingw64/mingw-w64-x86_64-fontconfig \
mingw64/mingw-w64-x86_64-librsvg \
mingw64/mingw-w64-x86_64-libxml2
mingw64/mingw-w64-x86_64-libxml2 \
mingw64/mingw-w64-x86_64-libgsf

# create .lib files for vc++

Expand Down
5 changes: 3 additions & 2 deletions test/canvas.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -934,8 +934,9 @@ describe('Canvas', function () {

ctx.textBaseline = "alphabetic"
var metrics = ctx.measureText("Alphabet")
// Zero if the given baseline is the alphabetic baseline
assert.equal(metrics.alphabeticBaseline, 0)
// Actual value depends on font library version. Have observed values
// between 0 and 0.769.
assert.ok(metrics.alphabeticBaseline >= 0 && metrics.alphabeticBaseline <= 1);
// Positive = going up from the baseline
assert.ok(metrics.actualBoundingBoxAscent > 0)
// Positive = going down from the baseline
Expand Down

0 comments on commit f0974bc

Please sign in to comment.