Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesign test suite #883

Closed
9 of 13 tasks
diptorupd opened this issue Jan 20, 2023 · 1 comment
Closed
9 of 13 tasks

Redesign test suite #883

diptorupd opened this issue Jan 20, 2023 · 1 comment
Assignees
Labels
examples tests Related to test suite or test run infrastructure

Comments

@diptorupd
Copy link
Contributor

diptorupd commented Jan 20, 2023

The numba-dpex test suite parameterized all unit tests on a set of filter strings. The result is a bulk of tests get skipped when unit tests are run a system that does not support all devices specified as filter strings.

For example, on a recent run of the public CI in github only ~600 tests ran out of ~2300. the rest got skipped.

The test suite needs to be redesigned:

  • Remove filter string parameter and instead always use the device returned by dpctl.select_default_device.
  • Control the default device using dpc++'s environment flags
  • Reorganize tests into directories to help run subset of test suite when needed. The test directory structure should be the same as the source directory layout.
  • Remove init.py inside test directories as we do not want tests to be importable modules.
  • Identify and remove duplicates
  • Change all kernel tests to follow compute follows data programming model #995
  • Refactor all tests that use NumPy arrays as arguments to kernels (later for njit once njit supports compute follows data)
  • Fix deprecation and other warnings
  • Each test must have a docstring explaining what the test is doing.
  • Add more Numba IR and LLVM IR level tests. Currently, most tests are end-to-end tests. We need more IR level tests especially as we start focusing on IR transformations and optimizations.
  • Every pass should have IR-level tests cases.
  • Remove or adapt all njit tests to be dpjit tests.
  • All tests with capturing stdout and stderr should print captured text on error #21
@chudur-budur chudur-budur added tests Related to test suite or test run infrastructure examples labels Apr 3, 2023
@diptorupd
Copy link
Contributor Author

Adding IR-level test cases is already being tracked in several other open issues. I think based on current status the issue can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
examples tests Related to test suite or test run infrastructure
Projects
None yet
Development

No branches or pull requests

2 participants