test: adds weekly scheduled smoke tests #28
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes/implements the following bugs/features
What
Adds weekly scheduled smoke tests to be run on every Sunday at 00:00. Tests can also be trigger manually (
workflow_dispatch
). These tests will be looking for rule crashes.Some example ESLint community plugins that are using this tool already:
eslint-plugin-react
eslint-plugin-jest
eslint-plugin-jest-extended
eslint-plugin-jest-dom
eslint-plugin-testing-library
eslint-plugin-unicorn
Helps finding issues like below even before releasing:
no-conditional-tests
when using array destructuring #21expect-expect
rule when calling function without assignment #20Here is the first run: AriPerkkio#1
How
Plugin maintainer making sure all existing rules do not crash
Utilizes https://github.com/marketplace/actions/eslint-remote-tester-runner. Each test run will check as many repositories as it can in 5 hours 30 minutes (default value of
timeLimit
). Utilizes over 10K repositories from eslint-remote-tester-repositories.Available configuration options:
eslint-remote-tester-run-action
: https://github.com/marketplace/actions/eslint-remote-tester-runner#action-parameterseslint-remote-tester
: https://github.com/AriPerkkio/eslint-remote-tester#configuration-optionsWhen test fails the @github-actions bot will open new issue with title
Results of weekly scheduled smoke test
. It will reuse the existing issue if previous one hasn't yet been closed. It's recommended to close the issues once their findings are fixed.Below are two examples of test results: