-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Query: add query metrics to calls going through the Store API #5741
Conversation
These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]>
…ent-reloader Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
Signed-off-by: Douglas Camata <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
…etrics Signed-off-by: Douglas Camata <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks 💪🏽 (:
pkg/store/telemetry/telemetry.go
Outdated
"series_le": strconv.Itoa(int(seriesLeBucket)), | ||
"samples_le": strconv.Itoa(int(samplesLeBucket)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My main concerns is still this: Is such "weird" metric usable in PromQL? (two dimensional histogram)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussed in #5741 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, it's actually len(series_le) * len(samples_le)
number of histograms.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I am fine then, thanks for explanation (:
pkg/store/telemetry/telemetry.go
Outdated
"series_le": strconv.Itoa(int(seriesLeBucket)), | ||
"samples_le": strconv.Itoa(int(samplesLeBucket)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, it's actually len(series_le) * len(samples_le)
number of histograms.
pkg/store/telemetry/telemetry.go
Outdated
"series_le": strconv.Itoa(int(seriesLeBucket)), | ||
"samples_le": strconv.Itoa(int(samplesLeBucket)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I am fine then, thanks for explanation (:
Signed-off-by: Douglas Camata <[email protected]>
0951370
Signed-off-by: Matej Gera <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(previously approved, approving after merging main
and merging this PR now)
…-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]>
…thanos-io#5741)" This reverts commit eec4fd0.
…thanos-io#5741)" This reverts commit eec4fd0. Signed-off-by: utukj <[email protected]>
…-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]>
…thanos-io#5741)" This reverts commit 7a77769. Signed-off-by: utukj <[email protected]>
…specify metric labels to collect (#5785) * added unit tests for long labels and no external labels Signed-off-by: utukj <[email protected]> * trimmed too long external labels Signed-off-by: utukj <[email protected]> * added optional label selection Signed-off-by: utukj <[email protected]> * added cmd flag for choosing metric labels Signed-off-by: utukj <[email protected]> * updated docs Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset.go full sentence fix Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * minor fixes from code review Signed-off-by: utukj <[email protected]> * fixed code comments Signed-off-by: utukj <[email protected]> * used enum for labels Signed-off-by: utukj <[email protected]> * updated query docs Signed-off-by: utukj <[email protected]> * cleaned up tests Signed-off-by: utukj <[email protected]> * Updates busybox SHA (#5793) Signed-off-by: GitHub <[email protected]> Signed-off-by: GitHub <[email protected]> Co-authored-by: yeya24 <[email protected]> Signed-off-by: utukj <[email protected]> * Receive: Reload tenant limit configuration on file change (#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <[email protected]> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <[email protected]> * Run goimports Signed-off-by: Douglas Camata <[email protected]> * Properly cancel the context in the test Signed-off-by: Douglas Camata <[email protected]> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <[email protected]> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <[email protected]> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <[email protected]> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <[email protected]> * Update lastReload Signed-off-by: Douglas Camata <[email protected]> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <[email protected]> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <[email protected]> * Dain reload timer before resetting Signed-off-by: Douglas Camata <[email protected]> * Run tests in parallel Signed-off-by: Douglas Camata <[email protected]> * Simplify debouncing logic Signed-off-by: Douglas Camata <[email protected]> * Add more tests to file reloader Signed-off-by: Douglas Camata <[email protected]> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <[email protected]> * Use absolute path to config file Signed-off-by: Douglas Camata <[email protected]> * Get rid of parallel test Signed-off-by: Douglas Camata <[email protected]> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <[email protected]> * Remove useless sleep Signed-off-by: Douglas Camata <[email protected]> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <[email protected]> * Remove unused fucntion Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright to test file Signed-off-by: Douglas Camata <[email protected]> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <[email protected]> * Wrap error when reloading config Signed-off-by: Douglas Camata <[email protected]> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <[email protected]> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <[email protected]> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <[email protected]> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <[email protected]> * Remove wrong nil guard Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: utukj <[email protected]> * Query: add query metrics to calls going through the Store API (#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]> * docs: mark me as shepherd for next release (#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: utukj <[email protected]> * Revert "docs: mark me as shepherd for next release (#5797)" This reverts commit ea646a6. Signed-off-by: utukj <[email protected]> * Revert "Query: add query metrics to calls going through the Store API (#5741)" This reverts commit eec4fd0. Signed-off-by: utukj <[email protected]> * Revert "Receive: Reload tenant limit configuration on file change (#5673)" This reverts commit 24e1cc0. Signed-off-by: utukj <[email protected]> * Revert "Updates busybox SHA (#5793)" This reverts commit 9474c00. Signed-off-by: utukj <[email protected]> * Updates busybox SHA (#5793) Signed-off-by: GitHub <[email protected]> Signed-off-by: GitHub <[email protected]> Co-authored-by: yeya24 <[email protected]> Signed-off-by: utukj <[email protected]> * Receive: Reload tenant limit configuration on file change (#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <[email protected]> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <[email protected]> * Run goimports Signed-off-by: Douglas Camata <[email protected]> * Properly cancel the context in the test Signed-off-by: Douglas Camata <[email protected]> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <[email protected]> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <[email protected]> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <[email protected]> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <[email protected]> * Update lastReload Signed-off-by: Douglas Camata <[email protected]> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <[email protected]> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <[email protected]> * Dain reload timer before resetting Signed-off-by: Douglas Camata <[email protected]> * Run tests in parallel Signed-off-by: Douglas Camata <[email protected]> * Simplify debouncing logic Signed-off-by: Douglas Camata <[email protected]> * Add more tests to file reloader Signed-off-by: Douglas Camata <[email protected]> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <[email protected]> * Use absolute path to config file Signed-off-by: Douglas Camata <[email protected]> * Get rid of parallel test Signed-off-by: Douglas Camata <[email protected]> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <[email protected]> * Remove useless sleep Signed-off-by: Douglas Camata <[email protected]> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <[email protected]> * Remove unused fucntion Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright to test file Signed-off-by: Douglas Camata <[email protected]> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <[email protected]> * Wrap error when reloading config Signed-off-by: Douglas Camata <[email protected]> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <[email protected]> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <[email protected]> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <[email protected]> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <[email protected]> * Remove wrong nil guard Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: utukj <[email protected]> * Query: add query metrics to calls going through the Store API (#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]> * docs: mark me as shepherd for next release (#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: utukj <[email protected]> * Revert "docs: mark me as shepherd for next release (#5797)" This reverts commit c509c0e. Signed-off-by: utukj <[email protected]> * Revert "Updates busybox SHA (#5793)" This reverts commit ad11a03. Signed-off-by: utukj <[email protected]> * Revert "Query: add query metrics to calls going through the Store API (#5741)" This reverts commit 7a77769. Signed-off-by: utukj <[email protected]> * Revert "Receive: Reload tenant limit configuration on file change (#5673)" This reverts commit 32ca327. Signed-off-by: utukj <[email protected]> * fixed lint issue Signed-off-by: utukj <[email protected]> * added unit test for truncate and clean up Signed-off-by: utukj <[email protected]> * fixed truncate label func and added more tests Signed-off-by: utukj <[email protected]> * removed name from truncate test Signed-off-by: utukj <[email protected]> * reorganized test cases and removed redundant comments Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * fixed failing checks Signed-off-by: utukj <[email protected]> * e2e: Adding test for querier with two stores loadbalancing across them. Signed-off-by: bwplotka <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * dumped long expected output in unittest Signed-off-by: utukj <[email protected]> * Revert "e2e: Adding test for querier with two stores loadbalancing across them." This reverts commit 96b1545. Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * moved label definition to endpointset Signed-off-by: utukj <[email protected]> Signed-off-by: utukj <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> Signed-off-by: GitHub <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: bwplotka <[email protected]> Co-authored-by: Bartlomiej Plotka <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: yeya24 <[email protected]> Co-authored-by: Douglas Camata <[email protected]> Co-authored-by: Matej Gera <[email protected]> Co-authored-by: Giedrius Statkevičius <[email protected]>
…specify metric labels to collect (thanos-io#5785) * added unit tests for long labels and no external labels Signed-off-by: utukj <[email protected]> * trimmed too long external labels Signed-off-by: utukj <[email protected]> * added optional label selection Signed-off-by: utukj <[email protected]> * added cmd flag for choosing metric labels Signed-off-by: utukj <[email protected]> * updated docs Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset.go full sentence fix Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * minor fixes from code review Signed-off-by: utukj <[email protected]> * fixed code comments Signed-off-by: utukj <[email protected]> * used enum for labels Signed-off-by: utukj <[email protected]> * updated query docs Signed-off-by: utukj <[email protected]> * cleaned up tests Signed-off-by: utukj <[email protected]> * Updates busybox SHA (thanos-io#5793) Signed-off-by: GitHub <[email protected]> Signed-off-by: GitHub <[email protected]> Co-authored-by: yeya24 <[email protected]> Signed-off-by: utukj <[email protected]> * Receive: Reload tenant limit configuration on file change (thanos-io#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <[email protected]> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <[email protected]> * Run goimports Signed-off-by: Douglas Camata <[email protected]> * Properly cancel the context in the test Signed-off-by: Douglas Camata <[email protected]> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <[email protected]> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <[email protected]> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <[email protected]> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <[email protected]> * Update lastReload Signed-off-by: Douglas Camata <[email protected]> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <[email protected]> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <[email protected]> * Dain reload timer before resetting Signed-off-by: Douglas Camata <[email protected]> * Run tests in parallel Signed-off-by: Douglas Camata <[email protected]> * Simplify debouncing logic Signed-off-by: Douglas Camata <[email protected]> * Add more tests to file reloader Signed-off-by: Douglas Camata <[email protected]> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <[email protected]> * Use absolute path to config file Signed-off-by: Douglas Camata <[email protected]> * Get rid of parallel test Signed-off-by: Douglas Camata <[email protected]> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <[email protected]> * Remove useless sleep Signed-off-by: Douglas Camata <[email protected]> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <[email protected]> * Remove unused fucntion Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright to test file Signed-off-by: Douglas Camata <[email protected]> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <[email protected]> * Wrap error when reloading config Signed-off-by: Douglas Camata <[email protected]> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <[email protected]> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <[email protected]> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <[email protected]> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <[email protected]> * Remove wrong nil guard Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: utukj <[email protected]> * Query: add query metrics to calls going through the Store API (thanos-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]> * docs: mark me as shepherd for next release (thanos-io#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: utukj <[email protected]> * Revert "docs: mark me as shepherd for next release (thanos-io#5797)" This reverts commit ea646a6. Signed-off-by: utukj <[email protected]> * Revert "Query: add query metrics to calls going through the Store API (thanos-io#5741)" This reverts commit eec4fd0. Signed-off-by: utukj <[email protected]> * Revert "Receive: Reload tenant limit configuration on file change (thanos-io#5673)" This reverts commit 24e1cc0. Signed-off-by: utukj <[email protected]> * Revert "Updates busybox SHA (thanos-io#5793)" This reverts commit 9474c00. Signed-off-by: utukj <[email protected]> * Updates busybox SHA (thanos-io#5793) Signed-off-by: GitHub <[email protected]> Signed-off-by: GitHub <[email protected]> Co-authored-by: yeya24 <[email protected]> Signed-off-by: utukj <[email protected]> * Receive: Reload tenant limit configuration on file change (thanos-io#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <[email protected]> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <[email protected]> * Run goimports Signed-off-by: Douglas Camata <[email protected]> * Properly cancel the context in the test Signed-off-by: Douglas Camata <[email protected]> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <[email protected]> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <[email protected]> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <[email protected]> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <[email protected]> * Update lastReload Signed-off-by: Douglas Camata <[email protected]> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <[email protected]> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <[email protected]> * Dain reload timer before resetting Signed-off-by: Douglas Camata <[email protected]> * Run tests in parallel Signed-off-by: Douglas Camata <[email protected]> * Simplify debouncing logic Signed-off-by: Douglas Camata <[email protected]> * Add more tests to file reloader Signed-off-by: Douglas Camata <[email protected]> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <[email protected]> * Use absolute path to config file Signed-off-by: Douglas Camata <[email protected]> * Get rid of parallel test Signed-off-by: Douglas Camata <[email protected]> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <[email protected]> * Remove useless sleep Signed-off-by: Douglas Camata <[email protected]> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <[email protected]> * Remove unused fucntion Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright to test file Signed-off-by: Douglas Camata <[email protected]> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <[email protected]> * Wrap error when reloading config Signed-off-by: Douglas Camata <[email protected]> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <[email protected]> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <[email protected]> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <[email protected]> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <[email protected]> * Remove wrong nil guard Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: utukj <[email protected]> * Query: add query metrics to calls going through the Store API (thanos-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]> * docs: mark me as shepherd for next release (thanos-io#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: utukj <[email protected]> * Revert "docs: mark me as shepherd for next release (thanos-io#5797)" This reverts commit c509c0e. Signed-off-by: utukj <[email protected]> * Revert "Updates busybox SHA (thanos-io#5793)" This reverts commit ad11a03. Signed-off-by: utukj <[email protected]> * Revert "Query: add query metrics to calls going through the Store API (thanos-io#5741)" This reverts commit 7a77769. Signed-off-by: utukj <[email protected]> * Revert "Receive: Reload tenant limit configuration on file change (thanos-io#5673)" This reverts commit 32ca327. Signed-off-by: utukj <[email protected]> * fixed lint issue Signed-off-by: utukj <[email protected]> * added unit test for truncate and clean up Signed-off-by: utukj <[email protected]> * fixed truncate label func and added more tests Signed-off-by: utukj <[email protected]> * removed name from truncate test Signed-off-by: utukj <[email protected]> * reorganized test cases and removed redundant comments Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * fixed failing checks Signed-off-by: utukj <[email protected]> * e2e: Adding test for querier with two stores loadbalancing across them. Signed-off-by: bwplotka <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * dumped long expected output in unittest Signed-off-by: utukj <[email protected]> * Revert "e2e: Adding test for querier with two stores loadbalancing across them." This reverts commit 96b1545. Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * moved label definition to endpointset Signed-off-by: utukj <[email protected]> Signed-off-by: utukj <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> Signed-off-by: GitHub <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: bwplotka <[email protected]> Co-authored-by: Bartlomiej Plotka <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: yeya24 <[email protected]> Co-authored-by: Douglas Camata <[email protected]> Co-authored-by: Matej Gera <[email protected]> Co-authored-by: Giedrius Statkevičius <[email protected]>
…specify metric labels to collect (thanos-io#5785) * added unit tests for long labels and no external labels Signed-off-by: utukj <[email protected]> * trimmed too long external labels Signed-off-by: utukj <[email protected]> * added optional label selection Signed-off-by: utukj <[email protected]> * added cmd flag for choosing metric labels Signed-off-by: utukj <[email protected]> * updated docs Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset.go full sentence fix Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * minor fixes from code review Signed-off-by: utukj <[email protected]> * fixed code comments Signed-off-by: utukj <[email protected]> * used enum for labels Signed-off-by: utukj <[email protected]> * updated query docs Signed-off-by: utukj <[email protected]> * cleaned up tests Signed-off-by: utukj <[email protected]> * Updates busybox SHA (thanos-io#5793) Signed-off-by: GitHub <[email protected]> Signed-off-by: GitHub <[email protected]> Co-authored-by: yeya24 <[email protected]> Signed-off-by: utukj <[email protected]> * Receive: Reload tenant limit configuration on file change (thanos-io#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <[email protected]> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <[email protected]> * Run goimports Signed-off-by: Douglas Camata <[email protected]> * Properly cancel the context in the test Signed-off-by: Douglas Camata <[email protected]> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <[email protected]> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <[email protected]> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <[email protected]> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <[email protected]> * Update lastReload Signed-off-by: Douglas Camata <[email protected]> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <[email protected]> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <[email protected]> * Dain reload timer before resetting Signed-off-by: Douglas Camata <[email protected]> * Run tests in parallel Signed-off-by: Douglas Camata <[email protected]> * Simplify debouncing logic Signed-off-by: Douglas Camata <[email protected]> * Add more tests to file reloader Signed-off-by: Douglas Camata <[email protected]> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <[email protected]> * Use absolute path to config file Signed-off-by: Douglas Camata <[email protected]> * Get rid of parallel test Signed-off-by: Douglas Camata <[email protected]> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <[email protected]> * Remove useless sleep Signed-off-by: Douglas Camata <[email protected]> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <[email protected]> * Remove unused fucntion Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright to test file Signed-off-by: Douglas Camata <[email protected]> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <[email protected]> * Wrap error when reloading config Signed-off-by: Douglas Camata <[email protected]> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <[email protected]> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <[email protected]> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <[email protected]> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <[email protected]> * Remove wrong nil guard Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: utukj <[email protected]> * Query: add query metrics to calls going through the Store API (thanos-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]> * docs: mark me as shepherd for next release (thanos-io#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: utukj <[email protected]> * Revert "docs: mark me as shepherd for next release (thanos-io#5797)" This reverts commit ea646a6. Signed-off-by: utukj <[email protected]> * Revert "Query: add query metrics to calls going through the Store API (thanos-io#5741)" This reverts commit eec4fd0. Signed-off-by: utukj <[email protected]> * Revert "Receive: Reload tenant limit configuration on file change (thanos-io#5673)" This reverts commit 24e1cc0. Signed-off-by: utukj <[email protected]> * Revert "Updates busybox SHA (thanos-io#5793)" This reverts commit 9474c00. Signed-off-by: utukj <[email protected]> * Updates busybox SHA (thanos-io#5793) Signed-off-by: GitHub <[email protected]> Signed-off-by: GitHub <[email protected]> Co-authored-by: yeya24 <[email protected]> Signed-off-by: utukj <[email protected]> * Receive: Reload tenant limit configuration on file change (thanos-io#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <[email protected]> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <[email protected]> * Run goimports Signed-off-by: Douglas Camata <[email protected]> * Properly cancel the context in the test Signed-off-by: Douglas Camata <[email protected]> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <[email protected]> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <[email protected]> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <[email protected]> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <[email protected]> * Update lastReload Signed-off-by: Douglas Camata <[email protected]> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <[email protected]> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <[email protected]> * Dain reload timer before resetting Signed-off-by: Douglas Camata <[email protected]> * Run tests in parallel Signed-off-by: Douglas Camata <[email protected]> * Simplify debouncing logic Signed-off-by: Douglas Camata <[email protected]> * Add more tests to file reloader Signed-off-by: Douglas Camata <[email protected]> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <[email protected]> * Use absolute path to config file Signed-off-by: Douglas Camata <[email protected]> * Get rid of parallel test Signed-off-by: Douglas Camata <[email protected]> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <[email protected]> * Remove useless sleep Signed-off-by: Douglas Camata <[email protected]> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <[email protected]> * Remove unused fucntion Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright to test file Signed-off-by: Douglas Camata <[email protected]> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <[email protected]> * Wrap error when reloading config Signed-off-by: Douglas Camata <[email protected]> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <[email protected]> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <[email protected]> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <[email protected]> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <[email protected]> * Remove wrong nil guard Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: utukj <[email protected]> * Query: add query metrics to calls going through the Store API (thanos-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <[email protected]> * Fix some linter warnings Signed-off-by: Douglas Camata <[email protected]> * Remove useless logs Signed-off-by: Douglas Camata <[email protected]> * Refactor query tests Signed-off-by: Douglas Camata <[email protected]> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <[email protected]> * Remove TODO comment Signed-off-by: Douglas Camata <[email protected]> * Fix query tests Signed-off-by: Douglas Camata <[email protected]> * Reformat query docs Signed-off-by: Douglas Camata <[email protected]> * Remove useless return Signed-off-by: Douglas Camata <[email protected]> * Put back old query docs Signed-off-by: Douglas Camata <[email protected]> * Update query docs again Signed-off-by: Douglas Camata <[email protected]> * Fix e2e env name Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Add missing copyright notice. Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <[email protected]> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <[email protected]> * Clean up test Signed-off-by: Douglas Camata <[email protected]> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <[email protected]> * Retrigger CI Signed-off-by: Douglas Camata <[email protected]> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <[email protected]> * Update changelog Signed-off-by: Douglas Camata <[email protected]> * Fix broken changelog after merge Signed-off-by: Douglas Camata <[email protected]> * Remove extra empty line Signed-off-by: Douglas Camata <[email protected]> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <[email protected]> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <[email protected]> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <[email protected]> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Co-authored-by: Matej Gera <[email protected]> Signed-off-by: utukj <[email protected]> * docs: mark me as shepherd for next release (thanos-io#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: utukj <[email protected]> * Revert "docs: mark me as shepherd for next release (thanos-io#5797)" This reverts commit c509c0e. Signed-off-by: utukj <[email protected]> * Revert "Updates busybox SHA (thanos-io#5793)" This reverts commit ad11a03. Signed-off-by: utukj <[email protected]> * Revert "Query: add query metrics to calls going through the Store API (thanos-io#5741)" This reverts commit 7a77769. Signed-off-by: utukj <[email protected]> * Revert "Receive: Reload tenant limit configuration on file change (thanos-io#5673)" This reverts commit 32ca327. Signed-off-by: utukj <[email protected]> * fixed lint issue Signed-off-by: utukj <[email protected]> * added unit test for truncate and clean up Signed-off-by: utukj <[email protected]> * fixed truncate label func and added more tests Signed-off-by: utukj <[email protected]> * removed name from truncate test Signed-off-by: utukj <[email protected]> * reorganized test cases and removed redundant comments Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * fixed failing checks Signed-off-by: utukj <[email protected]> * e2e: Adding test for querier with two stores loadbalancing across them. Signed-off-by: bwplotka <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * dumped long expected output in unittest Signed-off-by: utukj <[email protected]> * Revert "e2e: Adding test for querier with two stores loadbalancing across them." This reverts commit 96b1545. Signed-off-by: utukj <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> * moved label definition to endpointset Signed-off-by: utukj <[email protected]> Signed-off-by: utukj <[email protected]> Signed-off-by: Uwakmfon Utuk <[email protected]> Signed-off-by: GitHub <[email protected]> Signed-off-by: Douglas Camata <[email protected]> Signed-off-by: Matej Gera <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: bwplotka <[email protected]> Co-authored-by: Bartlomiej Plotka <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: yeya24 <[email protected]> Co-authored-by: Douglas Camata <[email protected]> Co-authored-by: Matej Gera <[email protected]> Co-authored-by: Giedrius Statkevičius <[email protected]>
|
||
// Aggregate is an aggregator for merging `storepb.SeriesStatsCounter` for each incoming fanned out query. | ||
func (s *seriesStatsAggregator) Aggregate(stats storepb.SeriesStatsCounter) { | ||
s.seriesStats.Series += stats.Series |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
doesn't number of series overlap too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does, but we are (or were back then) not tracking the what individual series was touched. It would be quite expensive to do the tracking.
@@ -168,6 +169,7 @@ func (g *GRPCAPI) QueryRange(request *querypb.QueryRangeRequest, srv querypb.Que | |||
request.EnableQueryPushdown, | |||
false, | |||
request.ShardInfo, | |||
query.NoopSeriesStatsReporter, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this set to no-op here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because this is an internally initiated request, not something started by a user.
just curious if we thought about expose these stats with the query response as well? |
Changes
Overall, this pull request's intent is to implement the first step in the proposal More granular query performance metrics.
Verification