Token Bucket Algorithm

Token Bucket Algorithm A workflow diagram generated by Archify. 01 / Token Bucket · Token supply 02 / Request handling path EX / Limit exceeded Refill · 10 tokens/sec · Token Bucket · Token supply Refill 10 tokens/sec Token Bucket · Max 100 tokens · Token Bucket · Token supply Token Bucket Max 100 tokens Request Arrives · Received by Envoy proxy · Request handling path Request Arrives Received by Envoy proxy Tokens Available? · Check remaining tokens · Request handling path Tokens Available? Check remaining tokens Allow Request · Consume 1 token · Request handling path Allow Request Consume 1 token Reject Request · Return 429 · Limit exceeded Reject Request Return 429 Decrease tokens Yes No Add 10 per second Legend Request intake Token handling Rejected Bucket state

EnvoyFilter token_bucket settings

  • • max_tokens: 100 — maximum tokens the bucket can hold
  • • tokens_per_fill: 10 / fill_interval: 1s — refills 10 per second
  • • When no token is left, Envoy returns 429

Characteristics of Local Rate Limiting

  • • Each Envoy proxy manages its own bucket independently
  • • 100 req/s per pod → 3 pods allow up to 300 req/s total
  • • Fast and simple, but cluster-wide accuracy is low