feat(cardinality): Implement Redis set based cardinality limiter

[Dav1dde]

Introduces a new relay-cardinality module, initially it is only housing the cardinality limiter, but at some point there may be more functionality related to cardinality (e.g. smart detection of high cardinality tags).

Implements a very basic cardinality limiter using one Redis set per organization and namespace to track cardinality.

API is still very much work in progress, this works for now but will definitely need to be changed in the future when the requirements for the cardinality limiter change. Outcomes, Tag erasure etc.

Epic: #2717

[Dav1dde]

I checked with godbolt this is optimized away when passing a vector already

[jjbayer]

I like the look of this! Just a bunch of random comments for now, will do a more in-depth review later.

[jjbayer]

nit: Why not check the flag outside of this function? Seems more straight-forward than passing a boolean that makes the function behave like a noop (but this might just be my personal preference).

[Dav1dde]

I implemented the function pretty much only for this check, so I can early return, otherwise it becomes very complex with the cfg(feature = "processing")

[jjbayer]

nit: Should make this function #[cfg(feature = "processing")]?

Suggested change

	_organization_id: u64,
	organization_id: u64,

[Dav1dde]

Unfortunately this is needed for non-processing, the entire function just exists to handle the if case with an early return to make the code somewhat readable and not a pure if else cfg mess.

[jjbayer]

Is it still a mess with the if_processing! macro?

[jjbayer]

Let's ship it! Before opting in all orgs we should transition from copying entire redis sets to a Lua script, as you suggested.

[jjbayer]

Not necessarily for this PR, but since this is a new crate, we should take a look at what make doc-rust produces and whether we're happy with the documentation. Other crates have a summary of what they do in the crate-level docs, e.g.

relay/relay-metrics/src/lib.rs

Lines 1 to 64 in a9e7ef5

	//! Metric protocol, aggregation and processing for Sentry.
	//!
	//! Metrics are high-volume values sent from Sentry clients, integrations, or extracted from errors
	//! and transactions, that can be aggregated and queried over large time windows. As opposed to rich
	//! errors and transactions, metrics carry relatively little context information in tags with low
	//! cardinality.
	//!
	//! # Protocol
	//!
	//! Clients submit metrics in a [text-based protocol](Bucket) based on StatsD. See the [field
	//! documentation](Bucket#fields) on `Bucket` for more information on the components. A sample
	//! submission looks like this:
	//!
	//! ```text
	#![doc = include_str!("../tests/fixtures/buckets.statsd.txt")]
	//! ```
	//!
	//! The metric type is part of its signature just like the unit. Therefore, it is allowed to reuse a
	//! metric name for multiple metric types, which will result in multiple metrics being recorded.
	//!
	//! # Metric Envelopes
	//!
	//! To send one or more metrics to Relay, the raw protocol is enclosed in an envelope item of type
	//! `metrics`:
	//!
	//! ```text
	//! {}
	//! {"type": "statsd", ...}
	#![doc = include_str!("../tests/fixtures/buckets.statsd.txt")]
	//! ...
	//! ```
	//!
	//! Note that the name format used in the statsd protocol is different from the MRI: Metric names
	//! are not prefixed with `<ty>:` as the type is somewhere else in the protocol. If no metric
	//! namespace is specified, the `"custom"` namespace is assumed.
	//!
	//! Optionally, a timestamp can be added to every line of the submitted envelope. The timestamp has
	//! to be a valid Unix timestamp (UTC) and must be prefixed with `T`. If it is omitted, the
	//! `received` time of the envelope is assumed.
	//!
	//! # Aggregation
	//!
	//! Relay accumulates all metrics in [time buckets](Bucket) before sending them onwards. Aggregation
	//! is handled by the [`Aggregator`], which should be created once for the entire system. It flushes
	//! aggregates in regular intervals, either shortly after their original time window has passed or
	//! with a debounce delay for backdated submissions.
	//!
	//! **Warning**: With chained Relays submission delays accumulate.
	//!
	//! Aggregate buckets are encoded in JSON with the following schema:
	//!
	//! ```json
	#![doc = include_str!("../tests/fixtures/buckets.json")]
	//! ```
	//!
	//! # Ingestion
	//!
	//! Processing Relays write aggregate buckets into the ingestion Kafka stream. The schema is similar
	//! to the aggregation payload, with the addition of scoping information. Each bucket is sent in a
	//! separate message:
	//!
	//! ```json
	#![doc = include_str!("../tests/fixtures/kafka.json")]
	//! ```

.

[Dav1dde]

I will definitely do this in a follow up, I also want to get a proper documentation in for how the limiter works!