feat(build): changes to decrease build time, cancel runs in case of multiple commits #5187
Conversation
anshbansal
force-pushed
the
parallelization-tests
branch
from
643bfb8 to
b6e71d1
Compare
anshbansal
changed the title
| @@ -321,7 +321,6 @@ def get_long_description(): | |||
| "pytest-asyncio>=0.16.0", | |||
| "pytest-cov>=2.8.1", | |||
| "pytest-docker>=0.10.3,<0.12", | |||
| "tox", | |||
There was a problem hiding this comment.
Also interested to know!
There was a problem hiding this comment.
tried parallelising via tox but there was some problem during builds. the build wheel being generated by the 3 test suites was stepping on each other's toes and causing at least one of them to fail. Also, thought it might be better to do it via github actions so we can easily parallelise and isolate. Now if that SAP HANA test flakes we can simply re-run that (that and nifi one under slow integration) one instead of all integration tests.
| @@ -15,6 +15,10 @@ on: | |||
| release: | |||
| types: [published, edited] | |||
|
|
|||
| concurrency: | |||
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} | |||
| cancel-in-progress: true | |||
There was a problem hiding this comment.
This means cancelling only those in progress for the same PR right?
There was a problem hiding this comment.
For the same branch
| @@ -24,6 +25,11 @@ | |||
| logger = logging.getLogger(__name__) | |||
|
|
|||
|
|
|||
| telemetry_enabled = ( | |||
There was a problem hiding this comment.
Why is this guy in here?
There was a problem hiding this comment.
Because in case we had file sink this was causing telemetry client id to fetched. In case telemetry is disabled it should not be doing that.
For unit tests this was doing 3 retries and failing. Adding this and passing the environment variable reduced time for unit tests by 30-60 secs.
Checklist