Agentic AI Platform on EKS Quiz
Twenty questions on current APIs, execution boundaries and validation limits.
1. What does PagedAttention primarily improve?
Answer and explanation
KV-cache block management and memory waste. It does not compress model weights or guarantee fixed 2–4x throughput.
2. How does an inference gateway differ from a training executor?
Answer and explanation
A supported gateway/plugin handles inference routing, access and limits. Trainer/Ray and other systems execute training. Gateway installation alone does not enable every security or A/B feature.
3. What must match in a RAG vector store?
Answer and explanation
Actual embedding dimension, model revision, normalization, metric and ingestion/query behavior. A tenant field alone is not isolation; enforce authenticated retrieval scope with server-side filters.
4. What control model does LangGraph provide?
Answer and explanation
Stateful nodes, edges, conditional routing and cycles. A graph alone does not implement tool execution, persistent state or bounded retries.
5. How do Langfuse and DCGM differ?
Answer and explanation
Langfuse tracks instrumented calls, traces, usage and evaluation; DCGM reports device metrics. Token-cost estimates are not invoices and differ from infrastructure measurements such as GPU temperature.
6. What API shape does Kagent 0.10.1 use?
Answer and explanation
kagent.dev/v1alpha2 with spec.type and declarative/BYO. Use declarative.modelConfig and McpServer/Agent tool references, not invented top-level llm, arbitrary python/eval tool definitions or permissions fields.
7. What isolation does time-slicing inside MIG provide?
Answer and explanation
Distinguish isolation between MIG instances from sharing inside one instance. Time-slice users within one instance do not gain new memory/fault isolation or proportional compute guarantees.
8. Does continuous batching mean requests never wait?
Answer and explanation
No. Scheduling can admit work each step but token budgets, KV capacity, concurrency and queue limits can cause waiting. Measure actual workload behavior.
9. When does chunk size 1000 mean 1000 tokens?
Answer and explanation
When the selected splitter measures tokens with the appropriate tokenizer. RecursiveCharacterTextSplitter defaults to character counts. Evaluate model limits, semantic units and retrieval quality together.
10. What must be checked for vLLM autoscaling?
Answer and explanation
Actual vllm: metric names, model/Pod labels, adapters, queue/latency signals and one scaling owner. Allocation is not GPU utilization; avoid competing HPA/KEDA control of the same replicas.
11. What does the KV cache store?
Answer and explanation
Key/value representations of earlier tokens to reduce repeated computation. It differs from response caching; GQA/MQA, sliding windows and other architectures change memory accounting.
12. How are traces and spans handled by the current Langfuse SDK?
Answer and explanation
Create child span/generation observations under one trace with start_as_current_observation and record usage_details. The inspected SDK 4.15.2 has no old trace()/generation() methods. Define secret/raw-data logging boundaries too.
13. Why is hybrid search more than two search calls?
Answer and explanation
It needs RRF or calibrated score fusion, equivalent authorization filters, deduplication and recall evaluation. Scores from different retrievers are not automatically comparable.
14. What was wrong with the original SqliteSaver example?
Answer and explanation
Use from_conn_string as a context manager. :memory: is not durable and a PostgreSQL DSN is not SQLite. Reading history alone does not replay/resume, and thread_id is not authentication.
15. What do TP and PP partition?
Answer and explanation
TP partitions operations within layers; PP partitions layer stages. Choose from architecture/backend/network/memory constraints, not universal powers of two or fixed GPUs per model size.
16. Exercise: what should a current vLLM deployment validate?
Answer and explanation
Use the pinned image/model, GPU/cache/startupProbe and ClusterIP example in the vLLM guide. Prepare namespace/drivers/devices. Schema checks without inference do not establish performance or availability.
17. Exercise: what should Langfuse deployment and Python tracing validate?
Answer and explanation
Use the current SDK example in the chapter, file credentials, span linkage and flush/shutdown. Chart 2.1.0 includes web/worker, Postgres, Valkey, object storage, ClickHouse and operator prerequisites. Default secret environment delivery differs from file-only policy.
18. Exercise: how should a RAG retry loop terminate safely without evidence?
Answer and explanation
Keep original question separate from search_query, set retry and recursion bounds, generate with evidence, rewrite within budget, then abstain when none is found. The chapter verifies these paths and SQLite reopening using local deterministic callbacks.
19. Advanced: what boundaries matter for a financial consultation agent?
Answer and explanation
Design authenticated tenant/account scope, retrieval permissions, provider egress, least-privilege/idempotent/approved tools, minimal sensitive logging, audit and human handoff. Do not overwrite an existing requires_human=true with false in a later branch. A compliance_check function name or LLM decision does not certify regulatory compliance.
20. Advanced: how should multi-model routing, A/B and cost optimization be validated?
Answer and explanation
Use provider adapters/credentials and reject when no candidate meets allowed-provider, budget and quality constraints. Implement stable A/B assignment, real routing consumers and measured guardrails. Cache keys include tenant/authorization/model/retrieval revisions; cost estimates include routing calls, retries, caches and GPU fixed costs. Invented savings percentages are not results.