test code is in unexpected places

[jkahn]

for example:
sourmash_lib/__init__.py is an unexpected place to find tests. Do they belong here, or can they be moved to a tests/test_*.py constellation?

Similarly, the files in sourmash_lib/ include test files, which are installed by pip; this is not appropriate for clients who are not developing the sourmash library itself. Probably, all of sourmash_tst_utils.py and sourmash_lib/test_*.py belong in a $repo/tests directory or they need to be renamed.

openjournals/joss-reviews#27

[ctb]

This is intentional! It can be helpful to have the tests around to debug installations.

[luizirber]

We are starting to have a sizable collection of tests, it is probably better to collect them in a tests/ directory now.

[ctb]

Counterpoint - right now the tests and test data are installed with sourmash, so (in theory) 'py.test --pyargs sourmash_lib' could work to test an installed version of sourmash. I like that idea (although note that it doesn't work right now, so, err, that should be fixed moving forward). If they are under 'tests/' then I do not think they should be installed systemwide. If they are under 'test_sourmash' or 'sourmash_test' (I kind of prefer the former) then maybe that's fine. I do find it annoying to have them under sourmash_lib/ but mainly for command line completion reasons. I know of no standard or requirement in Python that the tests not be packaged within the code.

[betatim]

pip install khmer followed by py.test --pyargs khmer runs tests. Doesn't this mean we can have the best of both worlds? Ship tests and keep them in a subdirectory for tab-completion heaven?

[ctb]

On Wed, Jan 18, 2017 at 07:28:17AM -0800, Tim Head wrote: `pip install khmer` followed by `py.test --pyargs khmer` runs tests. Doesn't this mean we can have the best of both worlds? Ship tests and keep them in a subdirectory for tab-completion heaven?

Fine, fine... at least one initial problem was that we didn't want to install 'tests' at the top import level in the python env. I think this is the key bit in khmer's setup.py that avoids that: "package_dir": {'khmer.tests': 'tests'}, and I think that's a fine solution here as well. PR if you agree and care enough ;)