Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[build][Jenkins] more efficient Jenkins CI, builds, improved linter c…
…onfig
This is a significant refactoring of the repo and CI.
1) Jenkins CI: more efficient, quicker builds
I propose that we adopt a "run often, run quickly" approach to our Jenkins CI.
ATM we mostly run Jenkins when it's time to release new Blueprint installers
(when a PR is merged on master). The other case is if "jenkins" is detected
in the PR branch or PR title, then almost a full Jenkins build is done, just
stopping short of publishing new Blueprint installers (full signing and
notarization is done, which can take a couple of hours).
I made it so we run a very light version of the Jenkins CI by default, that
builds the "dev" version of the app and pre-packages it using electron builder,
generating a directory that contains the app but no installers.
We really ought to run tests then, on the unpacked pre-packaged app, but
it will need to be done separately. Until then, we do not require the built-in
extensions (plugins), so I gave openvsx.org a rest and skip fetching them, by
default.
When it's detected that the Jenkins CI is processing the result of a merge on
master or if the new "release dry-run" mode is enabled [1], a production version
of the Blueprint apps will be produced, including built-in plugins, which will
then be packaged for "real", producing genuine OS-specific installers, signed
and notarized as required. This will take a lot longer to run, of course, but
may be required only a minority of the time.
Conclusion: with this PR, running Jenkins CI now takes about 12 minutes, and
since this is relatively quick, we can afford to run this when any PR branch
is created or updated.
Running the same CI in "release dry-run mode" takes about an hour. A release
will take longer, a bit over an hour, since additional steps are executed,
like copying the various installers to a release folder.
[1] About the "release dry-run" mode, it can be enabled in JenkinsFile,
"pipeline" definition near the top, in "environment":
BLUEPRINT_JENKINS_RELEASE_DRYRUN = 'true'
I wish I found a better mechanism to enable this "release dry-run" mode. But
this will serve for now. The "release dry-run" mode should be disabled before
merging a PR to master, but can in the meantime be used to test or troubleshoot
the whole pipeline, in particular signing and notarizing.
2) tsconfig/eslint configs:
Added configs, to improve linter coverage. This made it possible for some
source files, not previously covered, to get ts/linter feedback, both while
editing and when running `yarn lint`. This will help keep code in-line with
our standards. The config is not perfect and I would welcome further
improvements. But for now I think it's a nice improvement.
3) Build "scripts" (package.json)
Refactored the build commands ("scripts" section in package.json)
- previously, merely running 'yarn" in the repo's root would rebuild
every application from scratch. This prevents running a quick
"yarn install", e.g. just to re-install build dependencies
- the new version permits a granular build, with simple defaults
- inspired from a similar change not so long ago in the main repo
- see updated README for some examples
Other misc items:
- renamed extensions / applications
- made the browser application a first class citizen, equal to the
Electron application
- all applications now share a common 'plugins' folder rather than
each having their own. Moved the plugin-related entries to root
package.json
- to gain flexibility about which `yarn workspaces` are invoked for a
given `lerna` command, using the `--scope=` CLI option.I renamed the
repo's extensions and applications. This permits easily composing
commands that target only the extensions or only the applications.
e.g.:
``` json
"build:extensions": "lerna run --scope=\"blueprint*ext\" build",
"build:applications": "lerna run --scope=\"blueprint*app\" build --concurrency 1","
```
- renamed the extensions folders, made them more straightforward
- For systems with limited RAM, like on a Raspberry Pi 4B board with 4GB of RAM,
it's now possible to successfully build Blueprint. e.g. use the following cmd:
`yarn && yarn build:dev`, optionally followed by a packing command like:
`yarn electron package`
- (build:dev will build the Blueprint app in dev mode, which can be achieved
in 4GB RAM)
- [windows][jenkins] stash only dist folder: Currently, and only for Windows,
we stash the whole git repo, which is very big and takes long to stash and
un-stash. For the other OS, we only stash the dist folder, that contains the
platform-specific installer, that we built. Let's try that for Windows too,
and see if it works.
- [jenkins] Decrease timeout from 5h to 3h" Looking at the build history, all
recent builds that succeeded, did do under 2h30. OTOH, when a build hangs,
it needs to wait for the timeout to expire, wasting time at the current value
of 5h. Let's compromise at 3 hours and see how it goes?
- [jenkins][installer build] exclude browser app for now: We do not yet publish
the browser app, so let's skip building it to save time/resources. We may
revisit when we use the new browser app bundling, recently made available
upstream in the Theia framework.
Signed-off-by: Marc Dumais <[email protected]>- Loading branch information
1 parent
022878d
commit 6f65b08