Not yet implemented tests

[robbert-vdh]

There are quite a few test cases that would be great to have in clap-validator, but that just aren't implemented yet. This issue serves as a way to collect some of the useful (and less useful) tests that still need to be implemented. For implemented test cases, see PluginTestCase::description and PluginLibraryTestCase::description.

• Process random audio and note data through the plugin using in-place processing in accordance to the plugin's in-place pairs, assert that the outputs do not contain any non-finite or denormal values.
• (this test exists, but currently uses out-of-place processing) Send intentionally inconsistent and mismatching note and MIDI events to the plugin with its default parameter values and tests the output for consistency. Uses out-of-place audio processing.
• Process random audio and note data through the plugin configured at a fractional sample rate like 44123.45, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values.
• Process random audio and note data through the plugin configured at a sample rate of 10 kHz, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values.
• Process random audio and note data through the plugin configured at a sample rate of 384 kHz, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values.
• Process random audio and note data through the plugin configured with a maximum block size of 16384 samples, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values. These three tests can probably be combined into one test, with deactivate() calls in between.
• Process random audio and note data through the plugin configured with a maximum block size of 2048 samples, using different random block sizes between 1 and 2048 during several subsequent processing calls, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values.
• Process random audio and note data several times through the plugin with its parameters randomized between each processing call, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values. (this is a reminder to implement things like latency change requests when implementing tests relating to parameters)
• Process random audio and note data several times through the plugin with its parameters randomly snapped to either the minimum or the maximum value between each processing call, assert that the plugin has not modified the inputs and that the outputs do not contain any non-finite or denormal values.
• Process random audio through the plugin several times, estimate the average processing time, and then send denormal audio data to the plugin. Assert that the average processing time stays within the same order of magnitude. A similar test can be done for processing multiple plugin instances in parallel, but this test's reliability could be very dependent on the system.
• Repeatedly pump the operating system event loop on Windows and macOS in between each of the initialization and deinitialization steps to assert that the plugin does not crash when accessing data from an event handler without checking if that data has been initialized.
• The state and parameter reproducibility tests, but with saving and loading happening using different locales that use different decimal separators.
• Have the plugin load a megabyte of random data The plugin should error out and not crash.
• Some tests involving the audio-port-config extension.
• Some tests involving (mixed) 64-bit audio processing.
• Tests and dumping functionality for the preset discovery functionality.

The following tests for slight host misbehavior might also be useful:

• Provide zero audio input and output buffers but with non-null pointers and ask the plugin to process audio. This should not trigger a segfault.
• Provide events in the wrong order. Assert that the plugin doesn't crash during processing.

Some other CLAP features that aren't being tested right now include:

• The reset() function.
• The constant masks in the audio buffers.
• All of the extensions except for audio-ports and note-ports.