Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Percentiles #92

Open
jdhardy opened this issue Oct 24, 2017 · 2 comments · May be fixed by #93
Open

Percentiles #92

jdhardy opened this issue Oct 24, 2017 · 2 comments · May be fixed by #93

Comments

@jdhardy
Copy link

jdhardy commented Oct 24, 2017

Around here we commonly report latencies using percentiles (p99, etc.) so it would be very useful if the reports could be customized to include percentiles as well. I'm unsure if arbitrary ones should be allowed, or just hardcode the most common ones - p90, p95, p99, p99.9, p99.99 (p0, p50, and p100 are already included as min/median/max).
@ionelmc
Copy link
Owner

ionelmc commented Oct 24, 2017

Duly noted. I'll think about how to make it pluggable, 99.9(9) doesn't seem very common.

@jdhardy
Copy link
Author

jdhardy commented Oct 26, 2017

At least for me, 99.9 is the most important, so I'd like to see that one in there.

One option could be using --benchmark-columns=Min,Max,Median,P99.9 and just parsing anything that starts with 'P' into a percentile. A bit weird, but someone might have a use for weird ones.

I should be back on benchmarking next week so I may have a crack at this, since I'll need to report those numbers anyway.

@jdhardy jdhardy linked a pull request Nov 8, 2017 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Percentile Support jdhardy/pytest-benchmark
2 participants
@jdhardy @ionelmc