Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always run mpl comparison (currently it's only run when --mpl is set) #231

Open
velle opened this issue Sep 22, 2024 · 2 comments
Open

Always run mpl comparison (currently it's only run when --mpl is set) #231

velle opened this issue Sep 22, 2024 · 2 comments

Comments

@velle
Copy link

velle commented Sep 22, 2024

I expect the current design was chosen for a reason, but I don't quite understand. Please help me understand :)

Why is enabling pytest-mpl image comparison is not the default? Ie why is it only enabled when explicitly setting the --mpl flag?

Why are pytest-mpl tests logged as passed when the image comparison is in fact skipped?

I expect that just typing pytest is enough. I expect that this will run all tests, except some that are skipped because they can't be run, or don't make any sense to run, in the given context.

The current design carries a risk of the user incorrectly believing that all tests passed, when he in fact only executed a subset of the tests.

@velle
Copy link
Author

velle commented Sep 25, 2024

I consider using pytest-mpl as a plugin in a few tests on an open source project. But if I do that, then I will have to change both the docs and the github actions, to run pytest with the --mpl argument. Right? I don't exactly love that. But there might be a good reason, and if so, please share it with me :)

@rcomer
Copy link
Member

rcomer commented Sep 25, 2024

If you want to set default arguments for running pytest on your project, you can do so using the addopts setting in a configuration file:
https://docs.pytest.org/en/7.1.x/reference/customize.html#configuration-file-formats
https://docs.pytest.org/en/7.1.x/reference/reference.html?highlight=addopts#confval-addopts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants