Historical vLLM L4 measurement report

Historical vLLM L4 measurement report An architecture diagram generated by Archify. bench client Job · Python · ThreadPoolExecutor · Reported historical namespace setup bench client Job Python · ThreadPoolExecutor vllm-server Service · ClusterIP :8000 · Reported historical namespace setup vllm-server Service ClusterIP :8000 vLLM OpenAI server · v0.6.4.post1 · Qwen2.5-7B-Instruct · bf16 · Reported historical namespace setup vLLM OpenAI server v0.6.4.post1 · Qwen2.5-7B-Instruct · bf16 NVIDIA L4 GPU · 24GB · g6.2xlarge · Reported historical GPU NodePool NVIDIA L4 GPU 24GB · g6.2xlarge KV cache blocks · gpu-memory-utilization 0.9 · Reported historical GPU NodePool KV cache blocks gpu-memory-utilization 0.9 chat completion request 1..16 concurrent continuous batching block_size=16, max_model_len=4096 Reported historical namespace setup Reported historical GPU NodePool Legend Backend Database Security

Reported throughput

  • • Concurrency4:58.67 /8:109.04 /16:208.08tokens/s (client)
  • • Maximum server interval average:243tokens/s; not instantaneous peak

Reported latency

  • • p50:5.65s →7.52s (+33.1%), end-to-end HTTP
  • • Reported KV occupancy:up to2.6%; distinct from total VRAM

Evidence limits

  • • 2026-09-04 commit8622d388; raw logs/client artifact not found
  • • Bandwidth roofline is an estimate, not a profiled bottleneck.