-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: add provider verification CI job #326
Conversation
Codecov Report
@@ Coverage Diff @@
## master #326 +/- ##
==========================================
- Coverage 94.59% 94.56% -0.04%
==========================================
Files 27 27
Lines 2997 2998 +1
Branches 161 161
==========================================
Hits 2835 2835
- Misses 144 145 +1
Partials 18 18
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
.github/workflows/ci.yml
Outdated
- name: Publish Provider Verification Results | ||
if: matrix.python-version == '3.8' && matrix.toxenv == 'django32' | ||
run: | | ||
export PACT_BROKER_BASE_URL=https://edx.pactflow.io | ||
export PUBLISH_VERSION=`git rev-parse --short HEAD` | ||
export PUBLISH_VERIFICATION_RESULTS=true | ||
pytest edxval/pacts/verify_pact.py --ds=edxval.settings.pact |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This job should be run separately and not part of unit tests. In VEM, publish pact job is run with unit tests because the pact is created/updated during the unit test suite. The same is not true for verification. This job will read pact from the broker/local file and then run the verification.
4512b92
to
dfda6a7
Compare
.github/workflows/ci.yml
Outdated
|
||
|
||
provider-verification: | ||
name: Publish Provider Verification Results |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Pact Provider Verification. No need to add publish in start.
.github/workflows/ci.yml
Outdated
pip install -r requirements/ci.txt | ||
pip install -r requirements/test.txt | ||
|
||
- name: Publish Results |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: verify pacts
.github/workflows/ci.yml
Outdated
export PACT_BROKER_TOKEN=${{ secrets.PACT_FLOW_ACCESS_TOKEN }} | ||
export PUBLISH_VERSION=`git rev-parse --short HEAD` | ||
export PUBLISH_VERIFICATION_RESULTS=true | ||
pytest edxval/pacts/verify_pact.py --ds=edxval.settings.pact |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add -s
flag so that logs/console statements are visible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't that diable all the capturing? https://docs.pytest.org/en/6.2.x/capture.html#setting-capturing-methods-or-disabling-capturing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh. On local, using -s yielded in a lot more log output, so I thought it was giving more output. I didn't consult the documents.
cdcc32c
to
845b3ab
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- To add a link to Pending pacts document in PR description and why it is enabled
PROD-2552
Adds a separate pact provider verification job in CI. The pacts are retrieved from pactflow, verification is performed and after that, the results are published with dev/prod tags and provider version.
Also, we're allowing Pending Pacts in edx-val as not doing that may block the edxval from merging the new changes if one or more consumers pacts are failing (even when the provider is compatible with the deployed version of the consumer)