test: only skip validation tests when given explicit flag #472
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
acl-cqc
commented
Sep 10, 2024
| @@ -26,16 +26,17 @@ def get_validator() -> Path | None: | |||
| return None | |||
|
|
|||
|
|
|||
| @pytest.fixture | |||
| @pytest.fixture(scope="session") | |||
There was a problem hiding this comment.
session scope is not strictly necessary, but seems a good thing to have
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #472 +/- ##
==========================================
- Coverage 91.88% 91.42% -0.46%
==========================================
Files 58 59 +1
Lines 5887 6064 +177
==========================================
+ Hits 5409 5544 +135
- Misses 478 520 +42 ☔ View full report in Codecov by Sentry. |
acl-cqc
force-pushed
the
acl/pytest_no_validation
branch
from
f8291bc to
c1477c7
Compare
acl-cqc
changed the title
qartik
approved these changes
Sep 11, 2024
I see the CI does run execution tests with |
Co-authored-by: Kartik Singhal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pytest previously skipped the validation tests if the validator was not built, leading to the situation in #468 where many tests were accidentally skipped in CI.
Instead, fail the tests if the validator is not installed, but allow
pytest --no_validationto skip them instead.The possibility of doing similar for execution tests (somewhat-silently skipped if
execute-llvmisn't installed) is left for a later PR.