Skip to content

Commit

Permalink
CI: Refactor workflows (#1790)
Browse files Browse the repository at this point in the history
This is a minor reorganization of the new CI to be more modular and use
more organized patterns. Rather than one large all-in-one workflow,
items are now split slightly into discrete parts. Namely, there is a now
a PR workflow called for PRs, and a push workflow called for pushes to
the master branch, which in turn call constituent workflows for builds
and releases.

This enables a few conveniences:

- Workflows are split into concurrency groups that will automatically
cancel themselves. For example, if a PR is pushed multiple times in
succession, CI will be automatically canceled for all but the most
recent push.

- Permissions are applied discretely and in a somewhat more organized
way. The PR workflow simply has the "read" permission, while the master
push workflow has the "write" permission.

While the existing all-in-one workflow was not insecure, it relied on
GitHub Actions failsafes (such as pulls from public forks not being
granted write permissions nor secrets) in order to work properly, which
has the side effect of making the workflow's logic somewhat more
difficult to understand.

- New workflows can hopefully be more easily created based on
constituent parts. For example, the PR flow simply executes "build",
while the push workflow simply executes "build" as well as "release".
Future workflows could, for example, "build" and then call a new "test"
workflow to execute the test harnesses using the built artifacts and
report results. This can now be done without requiring as much delicate
attention to the existing workflow logic.

Future work:

- Ideally our "build" workflow could be refactored into a composite
action rather than a workflow. This would enable it to use defines
created in the initial "pull" or "push" workflows more easily, allowing
us to cleanly separate, for example, codesigning-related steps that
install secrets on the build machine, from actual build steps/jobs.

> [!NOTE]
> This was tested on a local fork, but several minor changes were made,
so it's opened as a draft until it can be double checked that nothing
broke.
  • Loading branch information
jcm93 authored Jan 27, 2025
1 parent c62e2a2 commit 8dbd213
Show file tree
Hide file tree
Showing 6 changed files with 125 additions and 82 deletions.
9 changes: 4 additions & 5 deletions .github/scripts/package_artifacts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ case ${GITHUB_REF} in
*) suffix="" ;;
esac

srcdir="${GITHUB_WORKSPACE}/src"
bindir="${GITHUB_WORKSPACE}/bin"

# Hack: Workaround for GitHub artifacts losing attributes.
Expand All @@ -20,8 +19,8 @@ do
outdir=ares${suffix}
mkdir ${outdir}
mkdir ${outdir}-dSYMs
cp -ar ${bindir}/ares-${package}-dSYMs/*.dSYM ${outdir}-dSYMs
cp -ar ${bindir}/ares-${package}/*.app ${outdir}
cp -a ${bindir}/ares-${package}-dSYMs/*.dSYM ${outdir}-dSYMs
cp -a ${bindir}/ares-${package}/*.app ${outdir}
zip -r ../ares-${package}.zip ${outdir}
zip -r ../ares-${package}-dSYMs.zip ${outdir}-dSYMs

Expand All @@ -37,10 +36,10 @@ do
outdir=ares${suffix}
mkdir ${outdir}
mkdir ${outdir}-PDBs
cp -ar ${bindir}/ares-${package}-PDBs/*.pdb ${outdir}-PDBs
cp -a ${bindir}/ares-${package}-PDBs/*.pdb ${outdir}-PDBs
zip -r ../ares-${package}-PDBs.zip ${outdir}-PDBs
rm -rf ${bindir}/ares-${package}/PDBs
cp -ar ${bindir}/ares-${package}/* ${outdir}
cp -a ${bindir}/ares-${package}/* ${outdir}
zip -r ../ares-${package}.zip ${outdir}
cd -
done
5 changes: 2 additions & 3 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ jobs:
certificatesha1: '${{ secrets.WIN_CERTHASH }}'
certificatename: '${{ secrets.WIN_CERTNAME }}'
folder: 'desktop-ui/out'
timestampUrl: 'http://timestamp.digicert.com'
timestampUrl: 'http://timestamp.digicert.com'
- name: "macOS: notarize"
if: false
continue-on-error: ${{ github.repository != 'ares-emulator/ares' }}
Expand Down Expand Up @@ -305,7 +305,7 @@ jobs:
- name: Upload ares-windows-msvc-arm64
uses: actions/upload-release-asset@v1
env: { GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}' }
with: { upload_url: '${{ env.UPLOAD_URL }}', asset_path: 'ares-windows-msvc-arm64.zip', asset_name: 'ares-windows-msvc-arm64.zip', asset_content_type: 'application/zip' }
with: { upload_url: '${{ env.UPLOAD_URL }}', asset_path: 'ares-windows-msvc-arm64.zip', asset_name: 'ares-windows-msvc-arm64.zip', asset_content_type: 'application/zip' }
- name: Upload ares-macos-universal
uses: actions/upload-release-asset@v1
env: { GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}' }
Expand All @@ -314,4 +314,3 @@ jobs:
# uses: actions/upload-release-asset@v1
# env: { GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}' }
# with: { upload_url: '${{ env.UPLOAD_URL }}', asset_path: 'ares-ubuntu.zip', asset_name: 'ares-ubuntu.zip', asset_content_type: 'application/zip' }

89 changes: 15 additions & 74 deletions .github/workflows/build_new.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
name: Build (New)
on:
push:
branches: [ '*' ]
pull_request:
branches: [ '*' ]
types: [ opened, synchronize, reopened ]
permissions:
contents: write
workflow_call:
inputs:
codesign:
description: Enable codesigning
required: false
default: false
type: boolean
jobs:
build:
strategy:
Expand Down Expand Up @@ -78,9 +78,9 @@ jobs:
msystem: ${{ matrix.platform.msystem }}
install: make git cmake ccache ninja ${{ matrix.platform.install }}
- name: "macOS: Import Certificate"
if: runner.os == 'macOS'
if: runner.os == 'macOS' && inputs.codesign
uses: apple-actions/import-codesign-certs@v3
continue-on-error: ${{ github.repository != 'ares-emulator/ares' || needs.pr-check.outputs.number != 'null' }}
continue-on-error: ${{ !inputs.codesign }}
with:
p12-file-base64: ${{ secrets.MACOS_CERTIFICATE_DATA }}
p12-password: ${{ secrets.MACOS_CERTIFICATE_PASSPHRASE }}
Expand Down Expand Up @@ -110,17 +110,8 @@ jobs:
CROSS_COMPILE: ${{ matrix.platform.native-cmake-preset != '' }}
NATIVE_PRESET: ${{ matrix.platform.native-cmake-preset }}
TARGET_PRESET: ${{ matrix.platform.target-cmake-preset }}
- name: "Build: macOS"
if: runner.os == 'macOS'
run: .github/scripts/build_macos.sh
env:
TARGET_PRESET: ${{ matrix.platform.target-cmake-preset }}
MACOS_CERTIFICATE_NAME: ${{ secrets.MACOS_CERTIFICATE_NAME }}
MACOS_NOTARIZATION_TEAMID: ${{ secrets.MACOS_NOTARIZATION_TEAMID }}
MACOS_KEYCHAIN_NAME: ares-macos-keychain
MACOS_KEYCHAIN_PASSWORD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
- name: "Windows: sign"
if: runner.os == 'Windows' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/'))
if: runner.os == 'Windows' && inputs.codesign
uses: DanaBear/code-sign-action@98c79121b376beab8d6a9484f445089db4461bca
with:
certificate: '${{ secrets.WIN_CERTIFICATE }}'
Expand All @@ -129,16 +120,12 @@ jobs:
certificatename: '${{ secrets.WIN_CERTNAME }}'
folder: 'build/desktop-ui/rundir/'
timestampUrl: 'http://timestamp.digicert.com'
- name: "macOS: notarize"
if: runner.os == 'macOS' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/'))
continue-on-error: ${{ github.repository != 'ares-emulator/ares' || needs.pr-check.outputs.number != 'null' }}
run: |
ditto -c -k --keepParent ${{ github.workspace }}/build/desktop-ui/RelWithDebInfo/ares.app /tmp/ares.zip
xcrun notarytool submit /tmp/ares.zip --apple-id "$MACOS_NOTARIZATION_USERNAME" --password "$MACOS_NOTARIZATION_PASSWORD" --team-id "$MACOS_NOTARIZATION_TEAMID" --wait
xcrun stapler staple ${{ github.workspace }}/build/desktop-ui/RelWithDebInfo/ares.app
- name: "Build: macOS"
if: runner.os == 'macOS'
run: .github/scripts/build_macos.sh
env:
MACOS_NOTARIZATION_USERNAME: ${{ secrets.MACOS_NOTARIZATION_USERNAME }}
MACOS_NOTARIZATION_PASSWORD: ${{ secrets.MACOS_NOTARIZATION_PASSWORD }}
TARGET_PRESET: ${{ matrix.platform.target-cmake-preset }}
MACOS_CERTIFICATE_NAME: ${{ secrets.MACOS_CERTIFICATE_NAME }}
MACOS_NOTARIZATION_TEAMID: ${{ secrets.MACOS_NOTARIZATION_TEAMID }}
- name: Upload Build (Windows)
if: runner.os != 'macOS' && runner.os != 'Linux'
Expand All @@ -164,49 +151,3 @@ jobs:
with:
name: ${{ matrix.program }}-${{ matrix.platform.name }}-dSYMs
path: build/dSYMs/*

release:
if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
needs:
- build
steps:
- uses: actions/checkout@v4
with:
path: 'src'
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
path: 'bin'
- name: Package Artifacts
run: src/.github/scripts/package_artifacts.sh
- name: Check Release Tag ☑️
id: check
run: |
if [[ ${GITHUB_REF_NAME} == master ]];
then
echo "tag=nightly" >> $GITHUB_OUTPUT
now=$(date +'%Y-%m-%d')
versionName="nightly ${now}"
echo "versionName=${versionName}" >> $GITHUB_OUTPUT
else
echo "tag=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
echo "versionName=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
fi
- name: Create Release 🛫
id: create_release
uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191
with:
draft: ${{ github.ref != 'refs/heads/master' }}
tag_name: ${{ steps.check.outputs.tag }}
name: ares ${{ steps.check.outputs.versionName }}
files: |
${{ github.workspace }}/ares-macos-universal.zip
${{ github.workspace }}/ares-macos-universal-dSYMs.zip
${{ github.workspace }}/ares-windows-x64.zip
${{ github.workspace }}/ares-windows-x64-PDBs.zip
${{ github.workspace }}/ares-windows-clang-cl-x64.zip
${{ github.workspace }}/ares-windows-clang-cl-x64-PDBs.zip
${{ github.workspace }}/ares-windows-clang-cl-arm64.zip
${{ github.workspace }}/ares-windows-clang-cl-arm64-PDBs.zip
17 changes: 17 additions & 0 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Pull Request
run-name: ${{ github.event.pull_request.title }} PR run
on:
pull_request:
branches: [ '*' ]
types: [ opened, synchronize, reopened ]
permissions:
contents: read
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
cancel-in-progress: true
jobs:
build-project:
name: Build
uses: ./.github/workflows/build_new.yml
permissions:
contents: read
24 changes: 24 additions & 0 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Push
on:
push:
branches: [ master ]
tags: [ 'v*' ]
concurrency:
group: '${{ github.workflow }} @ ${{ github.head_ref || github.ref }}'
cancel-in-progress: true
permissions:
contents: write
jobs:
build:
name: Build ares
uses: ./.github/workflows/build_new.yml
secrets: inherit
with:
codesign: true
release:
name: Release
needs: build
secrets: inherit
uses: ./.github/workflows/release.yml
with:
notarize: true
63 changes: 63 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Release
on:
workflow_call:
inputs:
notarize:
description: Notarize build (macOS only)
required: false
default: true
type: boolean
jobs:
release:
if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with:
path: 'src'
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
path: 'bin'
- name: "macOS: notarize"
if: inputs.notarize
run: |
ditto -c -k --keepParent ${{ github.workspace }}/bin/ares-macos-universal/ares.app /tmp/ares.zip
xcrun notarytool submit /tmp/ares.zip --apple-id "$MACOS_NOTARIZATION_USERNAME" --password "$MACOS_NOTARIZATION_PASSWORD" --team-id "$MACOS_NOTARIZATION_TEAMID" --wait
xcrun stapler staple ${{ github.workspace }}/bin/ares-macos-universal/ares.app
env:
MACOS_NOTARIZATION_USERNAME: ${{ secrets.MACOS_NOTARIZATION_USERNAME }}
MACOS_NOTARIZATION_PASSWORD: ${{ secrets.MACOS_NOTARIZATION_PASSWORD }}
MACOS_NOTARIZATION_TEAMID: ${{ secrets.MACOS_NOTARIZATION_TEAMID }}
- name: Package Artifacts
run: src/.github/scripts/package_artifacts.sh
- name: Check Release Tag ☑️
id: check
run: |
if [[ ${GITHUB_REF_NAME} == master ]];
then
echo "tag=nightly" >> $GITHUB_OUTPUT
now=$(date +'%Y-%m-%d')
versionName="nightly ${now}"
echo "versionName=${versionName}" >> $GITHUB_OUTPUT
else
echo "tag=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
echo "versionName=${GITHUB_REF_NAME}" >> $GITHUB_OUTPUT
fi
- name: Create Release 🛫
id: create_release
uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191
with:
draft: ${{ github.ref != 'refs/heads/master' }}
tag_name: ${{ steps.check.outputs.tag }}
name: ares ${{ steps.check.outputs.versionName }}
files: |
${{ github.workspace }}/ares-macos-universal.zip
${{ github.workspace }}/ares-macos-universal-dSYMs.zip
${{ github.workspace }}/ares-windows-x64.zip
${{ github.workspace }}/ares-windows-x64-PDBs.zip
${{ github.workspace }}/ares-windows-clang-cl-x64.zip
${{ github.workspace }}/ares-windows-clang-cl-x64-PDBs.zip
${{ github.workspace }}/ares-windows-clang-cl-arm64.zip
${{ github.workspace }}/ares-windows-clang-cl-arm64-PDBs.zip

0 comments on commit 8dbd213

Please sign in to comment.