AI/ML Best Practices Quiz
15 questions on measurement, recovery and current APIs.
1. What does TTFT measure, and when is it unavailable?
Answer and explanation
Time from request to first nonempty output. First HTTP frame and token may differ; a nonstreaming response cannot measure actual TTFT/ITL. Specify tokenizer, failures and warmup boundaries.
2. Does combining startup optimizations always improve by 80–95%?
Answer and explanation
No. Measure image fetch/unpack, model download/loading, readiness and node preparation separately. Evaluate prefetch/lazy-loading costs, caches and full-weight reads instead of fixed savings.
3. How should GPUs for large distributed training be selected?
Answer and explanation
Check exact instance-size devices/memory, CPU/RAM/network, model state/activations, communication, prices and quotas. A family name does not fix GPU counts or prove a universally best choice.
4. Are EFA and placement groups always required?
Answer and explanation
EFA is a high-performance path for suitable workloads, not every DDP prerequisite. EFA communication requires the same AZ; cluster placement groups are recommended for performance. Verify drivers/plugins/security groups/interfaces.
5. Does a dataset over10TB always require FSx?
Answer and explanation
No single size cutoff decides. Compare I/O, concurrency, metadata, latency, durability, mount semantics and cost. Distinguish EFS/FSx/S3/instance store and current gp3 limits.
6. Can temperature alone establish throttling or hardware failure?
Answer and explanation
No. Check device-specific limits, clocks, power, throttle reasons and workload. DCGMFB_USED is MiB and XID_ERRORS is a last-code gauge; not every XID means hardware failure.
7. Are 120 seconds grace and a/drain call sufficient for Spot inference?
Answer and explanation
EC2 does not always guarantee 120 seconds and an arbitrary vLLM/drain API cannot be assumed. Test gateway readiness, SIGTERM, streams, retries, duplicates and cache reloads.
8. How is mean ITL calculated?
Answer and explanation
With actual token timestamps and at least two tokens:(last-first)/(tokens-1). Account for multi-token chunks and empty/single-token outputs; distinguish the tool-specific TPOT definition.
9. What does a saturation test demonstrate?
Answer and explanation
How throughput, latency, errors and goodput change with load. It does not alone prove a CPU/GPU/memory bottleneck. Distinguish arrival rate from concurrency and inspect clients, profiling and queues.
10. What input does SOCI 0.15 standalone use?
Answer and explanation
A local OCI image-layout directory/archive, not a generic docker-save tar. convert --standalone needs no containerd; actual lazy-start benefits require runtime/registry/workload validation.
11. How is a 09–17 UTC business-hour budget configured?
Answer and explanation
Use 0 9 * * 1-5 with 8h duration.0 9-17 * * 1-5 starts hourly and extends through01:00nextday. Budgets limit voluntary disruption, not Spot reclamation, failures or forceful expiration.
12. Does ESO refresh automatically issue credentials, reload apps and audit every read?
Answer and explanation
No. Distinguish provider rotation, Secret synchronization and app rereads. subPath/env values do not automatically refresh; CloudTrail does not log every local read. ESO 2.10 serves v1, not v1beta1.
13. How should GPU memory for a 30B FP16 model be estimated?
Answer and explanation
Weights alone are about 60GB; add KV, activations, workspace and communication. Four 24GB GPUs total 96GB but require sharding/peak-memory/throughput checks.13B FP16 exceeds 24GB; 70B FP16 exceeds 96GB.
14. Does high current vLLM KV-cache occupancy always reject requests?
Answer and explanation
Use vllm:kv_cache_usage_perc. Interpret occupancy with queues, preemption and memory state; immediate rejection is not guaranteed. Avoid obsolete gpu_cache_usage_perc and unverified hit-rate names.
15. Is a Karpenter placement group configured through a tag?
Answer and explanation
Version 1.14.1 uses EC2NodeClass.spec.placementGroupSelector name/id. An aws:ec2:placement-group tag is not the placement API. Verify AZ, capacity and networking and assess single-AZ recovery risk.