[FR] global setup and teardown

[balshetzer]

Is your feature request related to a problem? Please describe.
I have benchmarks that need to create files. The benchmark is run multiple times to establish timing. But it's expensive and pointless to create and delete the files for each repetition.

Describe the solution you'd like
I expected setup to run once at the beginning and teardown to run once at the end. If that's not feasible then it would be good to have a real "global" setup/teardown that doesn't get run for each repetition.

[dmah42]

you can override main and do the global setup teardown in there.

[balshetzer]

There are lots of globals. I was referring to the setup and teardown for a specific benchmark. Right now the setup and teardown runs each time the benchmark is run even if it is a repetition or if it is during gathering timings for the "real" run.

[dmah42]

ah i see, i misunderstood "global".

so it sounds like there are three possible places to put setup/teardown:

1. main (global setup teardown)
2. per registered benchmark (not supported)
3. per benchmark run (fixture)

and you're looking for #2. i'm not sure implementing #2 would be generally a good idea, only because it would potentially cause users to make mistakes where they think what they're doing in #2 is actually #3 and they introduce cross-run dependencies.

[balshetzer]

What about a final teardown then? I could make my startup check if the files already exist and create them if they don't. But then not delete them until a final teardown.