vLLM Deployment Architecture

vLLM Deployment Architecture An architecture diagram generated by Archify. Client · HTTP requests · Networking Client HTTP requests Authenticated entry · internal Service / gateway · Networking Authenticated entry internal Service / gateway Kubernetes Service · selected replica / group · Networking Kubernetes Service selected replica / group vLLM Pod · example:1GPU / Qwen3-0.6B · Single Node Deployment vLLM Pod example:1GPU / Qwen3-0.6B vLLM Pod 0 · node-rank0 / API · Multi-Node Deployment vLLM Pod 0 node-rank0 / API NCCL Communication · GPU-to-GPU collectives · Multi-Node Deployment NCCL Communication GPU-to-GPU collectives vLLM Pod 1 · node-rank1 / headless · Multi-Node Deployment vLLM Pod 1 node-rank1 / headless Consistent model path · shared FS or per-node cache · Shared Storage Consistent model path shared FS or per-node cache Amazon S3 · Model repository · Shared Storage Amazon S3 Model repository inference request load model model download Job Networking Single Node Deployment Multi-Node Deployment Shared Storage Legend Networking vLLM Pod Shared Storage AWS storage Collective communication Client

Single-GPU example

  • • v0.29.0-cu129 amd64 digest + pinned Qwen revision
  • • ClusterIP / Recreate / startupProbe; not zero-downtime

Multi-node

  • • TP8 × PP2 illustrates one16GPU group.
  • • Restrict internal communication to a trusted network.