RFC: Rate-limiter API

[AndrewDryga]

The rate limit plugin requires us to follow one of the options:

First. To store a distributed CRDT counter which is merged via clustering protocol on an event or timing basis.

Pros:

• Each node is able to rate limit request even when split brain happens, counters then re-joined when nodes see each other again.
• Low latency overhead, since all calls are local.
• Annon users would benefit from using load balancers that route requests to the same gateway instance for a single client (eg. by balancing on IP or auth header). This CRDT would be local and only replication would be involved.

Cons:

• If all request are distributed over the whole cluster, memory consumption grows quickly (=NODES*CONSUMERS) with each new API client.
• Increased network traffic due to CRDT synchronization.
• We can provide only estimated rate limit - when split brain happens two separate counters would co-exist allowing to send more requests; ttl for counters is not 100% reliable, some counters could be reset before other ones (leverage generation counters?).
• Need to implement CRDT and gossip protocol.

--

Second. Build a hash ring out of all known Annon nodes and route counter increment calls or whole request to the node which is responsible for limiting it (probably via RPC calls; Kademilla DHT?).

Pros:

• Annon users would benefit from using load balancers that route requests to the same gateway instance for a single client (eg. by balancing on IP or auth header).
• Less memory consumption, only one counter for each API client.

Cons:

• Latency hit for rate-limited requests because of RPC call
• Integration with load balancer would require it to use exactly the same hashing as Annon, which is not feasible for most cases

--

Third. Provide only very limited rate limiting which would work either with load balancer that supports sticky sessions (route request to the same Annon instance for each consumer) or when there is only one Annon instance at all.

This is similar to option one, except we don't need to sync CRDT's.

--

Fourth. Use persistent backends.

Proc:

• Exact rate limits is guaranteed.
• Offload most of the development to integrating third-party service (eg. redis)

Cons:

• Split brain would result in downtime;
• Storage would result in downtime;
• Possibly we would require more infrastructure dependencies.

[AndrewDryga]

Kong is providing QOS levels with different backends for rate limiting - https://getkong.org/plugins/rate-limiting/.

[AndrewDryga]

http://blog.00null.net/post/166149523731/rate-limiting-phoenix-with-redis-too?utm_campaign=elixir_radar_116&utm_medium=email&utm_source=RD+Station

[AndrewDryga]

‪ https://github.com/myntra/golimit/blob/master/README.md‬