@pytest.mark.parameterize gives misleading error messages

[edschofield]

The following code fails with a misleading error message:

%%run_pytest[clean]
import pytest

@pytest.mark.parameterize('test_input,expected', [("3+5", 8)])
def test_eval(test_input, expected):
    assert eval(test_input) == expected

The error message is:

E                                                                        [100%]
==================================== ERRORS ====================================
________________ ERROR at setup of test_temperature_conversion _________________
file <ipython-input-22-dcba670f399e>, line 3
  @pytest.mark.parameterize("test_input,expected",
                            [
                                (100, 212)
                            ]
                           )
  def test_temperature_conversion(test_input, expected):
E       fixture 'test_input' not found
>       available fixtures: _session_faker, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, dash_br, dash_duo, dash_process_server, dash_thread_server, dashr, dashr_server, doctest_namespace, faker, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

The error message is misleading because the cause is actually that parameterize has a different spelling to the parametrize spelling expected by pytest.

This is an easy mistake to make and, with the above error, it can be difficult to understand how to solve the problem. According to various dictionaries (e.g. https://www.merriam-webster.com/dictionary/parameterize), the verbs "parameterize" and "parametrize" are variants. A quick Google search turns up 1.97 million hits for "parameterize" versus 763,000 hits for "parametrize".

For reference, using pytest by itself (without ipytest) gives a much more useful error message:

test_eval.py:3: in <module>
    @pytest.mark.parameterize('test_input,expected', [("3+5", 8)])
E   Failed: Unknown 'parameterize' mark, did you mean 'parametrize'?

[chmp]

Thanks for reprorting this issue with such detail (I really like the focus on the user perspective). Off the top of my head, I have no idea what could be the issue. I'll investigate.

[chmp]

Not sure, there is anything I can do about it. I will leave the issue open for now. Maybe something will come to mind.

The reason for this behavior is the following:

• The "MarkGenerator" is configured when the mark plugin is loaded (see here)
• When the marks are accessed, the MarkGenerator checks for known marks, but only if it was configured (see here)
• IPytest only collects the main module. However, since the IPython interpreter runs outside pytest, the MarkGenerator is not configured, when the test functions are defined

Put differently: the normal pytest the failure appears, when the test function is defined. In ipytest the failure is only caught, when the test is executed.