Skip to content

Commit

Permalink
ref(store): Random Kafka partitioning for sessions (#1194)
Browse files Browse the repository at this point in the history
Instead of using the session ID as partition key for producing sessions, assign
a random partitioning key. This ensures more even distribution of sessions on
the Kafka topic, even if clients are sending many updates for the same session.

We have noticed intermittent imbalance on the partitions. The theory is that
clients are sending repeat updates for the same session (ID) when a high number
of errors are occurring within that session. Every session update would go to
the same partition unless we use random partitioning.
  • Loading branch information
jan-auer authored Mar 10, 2022
1 parent 8dc32ab commit 6c3415c
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 3 deletions.
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
**Internal**:

- Spread out metric aggregation over the aggregation window to avoid concentrated waves of metrics requests to the upstream every 10 seconds. Relay now applies jitter to `initial_delay` to spread out requests more evenly over time. ([#1185](https://github.com/getsentry/relay/pull/1185))
- Add new statsd metrics for bucketing efficiency ([#1199](https://github.com/getsentry/relay/pull/1199), [#1192](https://github.com/getsentry/relay/pull/1192), [#1200](https://github.com/getsentry/relay/pull/1200))
- Use a randomized Kafka partitioning key for sessions instead of the session ID. ([#1194](https://github.com/getsentry/relay/pull/1194))
- Add new statsd metrics for bucketing efficiency. ([#1199](https://github.com/getsentry/relay/pull/1199), [#1192](https://github.com/getsentry/relay/pull/1192), [#1200](https://github.com/getsentry/relay/pull/1200))

## 22.2.0

Expand Down Expand Up @@ -53,6 +54,7 @@
- Metrics extraction config, custom tags. ([#1141](https://github.com/getsentry/relay/pull/1141))
- Update the user agent parser (uap-core Feb 2020 to Nov 2021). This allows Relay and Sentry to infer more recent browsers, operating systems, and devices in events containing a user agent header. ([#1143](https://github.com/getsentry/relay/pull/1143), [#1145](https://github.com/getsentry/relay/pull/1145))
- Improvements to Unity OS context parsing ([#1150](https://github.com/getsentry/relay/pull/1150))

**Bug Fixes**:

- Support Unreal Engine 5 crash reports. ([#1132](https://github.com/getsentry/relay/pull/1132))
Expand Down
4 changes: 2 additions & 2 deletions relay-server/src/actors/store.rs
Original file line number Diff line number Diff line change
Expand Up @@ -673,8 +673,8 @@ impl KafkaMessage {
Self::Attachment(message) => message.event_id.0,
Self::AttachmentChunk(message) => message.event_id.0,
Self::UserReport(message) => message.event_id.0,
Self::Session(message) => message.session_id,
Self::Metric(_message) => Uuid::nil(), // TODO(ja): Determine a partitioning key
Self::Session(_message) => Uuid::nil(), // Explicit random partitioning for sessions
Self::Metric(_message) => Uuid::nil(), // TODO(ja): Determine a partitioning key
Self::ProfilingTrace(_message) => Uuid::nil(), // TODO(pierre): parse session_id as uuid
Self::ProfilingSession(_message) => Uuid::nil(), // TODO(pierre): parse session_id as uuid
};
Expand Down

0 comments on commit 6c3415c

Please sign in to comment.