Pytest 3.3: tests being skipped without failing

[argaen]

After updating to pytest==3.3, some tests of our pipeline started to be skipped while in previous versions where being run without problems.

Haven't seen anything relevant enough in the release notes and this is an important since skipping tests may lead to deploy broken stacks to production...

In our case, it's due to using test_client fixture from pytest-aiohttp:

async def test_whatever(test_client):
    pass

16:47 $ pytest test_test.py -svvv
============================================================================================================================= test session starts ==============================================================================================================================
platform darwin -- Python 3.5.2, pytest-3.3.0, py-1.5.2, pluggy-0.6.0 -- /Users/manuelmiranda/.pyenv/versions/3.5.2/envs/play/bin/python
cachedir: .cache
rootdir: /Users/manuelmiranda/Projects/testproject, inifile:
plugins: mock-1.6.3, cov-2.5.1, asyncio-0.8.0, aiohttp-0.1.3
collected 1 item

test_test.py::test_whatever[loop0] SKIPPED                                                                                                                                                                                                                               [100%]

========================================================================================================================== 1 skipped in 0.01 seconds ===========================================================================================================================

Environment:

pytest==3.3.0
pytest-aiohttp=0.1.3

The fixture from pytest-aiohttp can be found in https://github.com/aio-libs/aiohttp/blob/master/aiohttp/pytest_plugin.py#L264. Maybe it is the fixture that is doing something wrong. If that's the case lets open an issue there but I think it would be useful a big disclaimer in the new release notes regarding this behavior change

[argaen]

Closing in favor of #2959 which are the same