Integrate nonius to provide more advanced benchmarking #1616
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #1616 +/- ##
========================================
Coverage ? 83.9%
========================================
Files ? 123
Lines ? 3496
Branches ? 0
========================================
Hits ? 2933
Misses ? 563
Partials ? 0 |
Codecov Report
@@ Coverage Diff @@
## master #1616 +/- ##
==========================================
+ Coverage 81.05% 83.67% +2.62%
==========================================
Files 124 123 -1
Lines 3478 3380 -98
==========================================
+ Hits 2819 2828 +9
+ Misses 659 552 -107 |
fodinabor
changed the title
Changes done to Nonius: * Moved things into "Catch::Benchmark" namespace * Benchmarks were integrated with `TEST_CASE`/`SECTION`/`GENERATE` macros * Removed Nonius's parameters for benchmarks, Generators should be used instead * Added relevant methods to the reporter interface (default-implemented, to avoid breaking existing 3rd party reporters) * Async processing is guarded with `_REENTRANT` macro for GCC/Clang, used by default on MSVC * Added a macro `CATCH_CONFIG_DISABLE_BENCHMARKING` that removes all traces of benchmarking from Catch
* Units from <ratio> are no longer redeclared in our own namespace * The default clock is `steady_clock`, not `high_resolution_clock`, because, as HH says "high_resolution_clock is useless. If you want measure the passing of time, use steady_clock. If you want user friendly time, use system_clock". * Benchmarking support is opt-in, not opt-out, to avoid the large (~10%) compile time penalty. * Benchmarking-related options in CLI are always present, to decrease the amount of code that is only compiled conditionally and making the whole shebang more maintainble.
Also updated baselines
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Integrates nonius benchmark library to catch to make benchmarks a good bit more advanced.
This implementation nearly works as a drop-in replacement for the old
BENCHMARKmacro - just a;needs to be added - and adds a whole lot of improvements:I guess this is a appropriate fix for #852 and also resolves #1186.
Left tasks/questions:
internal/benchmark/catch_constructor.hppis really needed, or I should remove it?(juint has been supported by nonius, so for example output see there..)To collect feedback about the reported info, here's some output from the
Benchmark.tests.cpp: