feat(extrapolation): add extrapolation develop docs #11780
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
2 Skipped Deployments
|
| | max | no | | ||
| | percentiles | yes | |
There was a problem hiding this comment.
min and max are percentiles, yet they say no. The subtle difference is that we do extrapolate values of counts and sums, but all of the other data is merely weighted. The sentence below alludes to this, but I think we might want to classify this into two categories for the future; technically there could be ways to extrapolate count_unique and maybe even maxima/minima.
| sidebar_order: 5 | ||
| --- | ||
|
|
||
| Sentry’s system uses sampling to reduce the amount of data ingested, for reasons of both performance and cost. When configured, Sentry only ingests a fraction of the data according to the specified sample rate of a project: if you sample at 10% and initially have 1000 requests to your site in a given timeframe, you will only see 100 spans in Sentry. Without making up for the sample rate, any metrics attached to these spans will misrepresent the true volume of the application. When different parts of the application have different sample rates, there will even be a bias towards some of them, skewing the total volume towards parts with higher sample rates. This effect is exacerbated for numerical attributes like latency, whose accuracy will be negatively affected by such a bias. To account for this fact, Sentry uses extrapolation to smartly combine the data to account for sample rates. |
There was a problem hiding this comment.
| Sentry’s system uses sampling to reduce the amount of data ingested, for reasons of both performance and cost. When configured, Sentry only ingests a fraction of the data according to the specified sample rate of a project: if you sample at 10% and initially have 1000 requests to your site in a given timeframe, you will only see 100 spans in Sentry. Without making up for the sample rate, any metrics attached to these spans will misrepresent the true volume of the application. When different parts of the application have different sample rates, there will even be a bias towards some of them, skewing the total volume towards parts with higher sample rates. This effect is exacerbated for numerical attributes like latency, whose accuracy will be negatively affected by such a bias. To account for this fact, Sentry uses extrapolation to smartly combine the data to account for sample rates. | |
| Sentry’s system uses sampling to reduce the amount of data ingested, for reasons of both performance and cost. When configured, Sentry only ingests a fraction of the data according to the specified sample rate of a project: if you sample at 10% and initially have 1000 requests to your site in a given timeframe, you will only see 100 spans in Sentry. Without making up for the sample rate, any metrics derived from these spans will misrepresent the true volume of the application. When different parts of the application have different sample rates, there will even be a bias towards some of them, skewing the total volume towards parts with higher sample rates. This effect is exacerbated for numerical attributes like latency, the accuracy of which will be negatively affected by such a bias. To account for this fact, Sentry uses extrapolation to smartly combine the data to account for sample rates. |
There was a problem hiding this comment.
The passive voice "the accuracy of which" is a bit cumbersome, replaced the sentence with this version: "This bias especially impacts numerical attributes like latency, reducing their accuracy."
| - **Accuracy** refers to data being correct. For example, the measured number of spans corresponds to the actual number of spans that were executed. As sample rates decrease, accuracy also goes down, because minor random decisions can influence the result in major ways. | ||
| - **Expressiveness** refers to data being able to express something about the state of the observed system. Expressiveness refers to the usefulness of the data for the user in a specific use case. | ||
|
|
||
| Data can be any combination of accurate and expressive. To illustrate these properties, let's look at some examples. A single sample with specific tags and a full trace can be very expressive, and a large amount of spans can have very misleading characteristics that are not very expressive. When traffic is low and 100% of data is sampled, the system is fully accurate despite aggregates being affected by inherent statistical uncertainty that reduce expressiveness. |
There was a problem hiding this comment.
This section reads very well now, thanks for restructuring. The example given for expressiveness is good, but it discusses the opposite of what is important for extrapolation: The expressiveness of aggregates, as in: when is it valid to deal with the aggregate and can you derive meaningful insights from it. This is not the case when the aggregate includes an insufficient amount of data points (even with high sample rates). Is "expressiveness" maybe the wrong word for this?
Lower in the document, "Sample Mode" discusses this scenario, and I think it's worth lifting this concern up here.
|
|
||
| Depending on the context and the use case, one mode may be more useful than the other. | ||
|
|
||
| Generally, default mose is useful for all queries that aggregate on a dataset of sufficient volume. As absolute sample size decreases below a certain limit, default mode becomes less and less expressive. There may be scenarios where the user will want to switch between modes, for example to examine the aggregate numbers first, and dive into single samples for investigation, therefore the extrapolation mode setting should be a transient view option that resets to default mode when the user opens the page the next time. |
There was a problem hiding this comment.
I was thinking about this a bit more, the listed example might not make sense if the product always shows matching samples next to the aggregates. Still, it can be useful of looking at the number of samples -- though what's a compelling use case to illustrate this? The usual example is "search by trace ID in trace explorer", but this is an extreme edge case.
| Generally, default mose is useful for all queries that aggregate on a dataset of sufficient volume. As absolute sample size decreases below a certain limit, default mode becomes less and less expressive. There may be scenarios where the user will want to switch between modes, for example to examine the aggregate numbers first, and dive into single samples for investigation, therefore the extrapolation mode setting should be a transient view option that resets to default mode when the user opens the page the next time. | |
| Generally, default mode is useful for all queries that aggregate on a dataset of sufficient volume. As absolute sample size decreases below a certain limit, default mode becomes less and less expressive. There are scenarios where the user needs to temporarily switch between modes, for example to examine the aggregate numbers first, and dive into single samples for investigation. Therefore, the extrapolation mode setting should be a transient view option that resets to default mode when the user opens the page the next time. |
(currently still in draft mode)
Closes getsentry/sentry#79815