QCheck2 shrinker improvements to list, array, bytes, strings, and functions #319
+575
−199
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR takes a first step towards improving the QCheck2 list shrinker.
The current one uses the input size tree and offers only prefixes, e.g., shrinking [1;2;3;4;5;6;7;8] to [], [1;2;3;4], [1;2;3;4;5;6], [1;2;3;4;5;6;7] (or something like that). This results in needlessly large counterexamples, if the property is essentially "lists do not contain the int 7". 😬
The PR replaces the list shrinker with a recursive one instead shrinking [1;2;3;4;5;6;7;8] to
[1;2;3;4], [5;6;7;8], [2;3;4;5;6;7;8] or [1;2;3;4;6;7;8], i.e.,
The shrinker then continues size reduction recursively from either of these, and switches to element shrinking afterwards.
The approach is not perfect, but it is a good step forward over the current status, and together with #318 it gets QCheck2 working better (witness the differences in expect test outputs).
Secondly the PR switches the other list, array, bytes, string #157, and function generators/shrinkers #163 to use the improved list shrinker, so that these benefit from the improvement too.
The PR does not improve the
QCheck2.Gen.{list_size, array_size, bytes_size, string_size}
generators.These take a size generator parameter, and I've not figured out how to cleanly respect that (read: preserve invariants), e.g., if a user provides a size generator of "even numbers", "multiples of 7", or "prime numbers". As such,
the shrinking behaviour and performance of those remain untouched.
QCheck2 still fares badly on < ocaml.5.0 because it needs
Random.State.split
and on OCaml4 that builds on a hack - that runs pretty slowly (on OCaml 5+ it works nice and fast). I have another patch to replace the hack with a faster one, that I'll submit as a separate PR.The PR adds a bunch of QCheck2 shrinker unit tests to document the effect of the change.
Should anyone wish to, the PR can thus be read commit-by-commit as gradual improvements.
I'm keen to hear from any QCheck2 users if this helps them (polite ping to @vch9 and friends)