SGLang Core Technology

SGLang Core Technology An architecture diagram generated by Archify. SGLang DSL · Programming interface · Frontend SGLang DSL Programming interface OpenAI Compatible API · HTTP endpoint · Frontend OpenAI Compatible API HTTP endpoint Native API · Frontend Native API RadixAttention · Radix tree-based KV cache · Runtime Engine RadixAttention Radix tree-based KV cache Grammar backend · XGrammar / Outlines / Llguidance · Runtime Engine Grammar backend XGrammar / Outlines / Llguidance Chunked Prefill · Runtime Engine Chunked Prefill FlashInfer Kernels · Optimized attention kernels · Runtime Engine FlashInfer Kernels Optimized attention kernels KV Cache Reuse · KV for identical common prefixes · Optimization KV Cache Reuse KV for identical common prefixes Schedule Overlapping · Optimization Schedule Overlapping Data Parallelism · Optimization Data Parallelism run program request cache sharing scheduler coordination Frontend Runtime Engine Optimization Legend Frontend APIs Runtime engine components Optimization techniques

Effects and limits

  • • An identical middle substring alone does not enable KV reuse.
  • • Schemas constrain format; they do not establish factual correctness.
  • • Do not assume fixed10x performance or optimal behavior on every device.