AI/ML Best Practices on EKS and Expected Outcomes

AI/ML Best Practices on EKS and Expected Outcomes An architecture diagram generated by Archify. Container Optimization · image size · layers · AI/ML Best Practices on EKS Container Optimization image size · layers GPU Selection · instance type fit · AI/ML Best Practices on EKS GPU Selection instance type fit Benchmarking & Performance · latency · throughput · AI/ML Best Practices on EKS Benchmarking & Performance latency · throughput Networking · RDMA · topology · AI/ML Best Practices on EKS Networking RDMA · topology Storage · model artifact I/O · AI/ML Best Practices on EKS Storage model artifact I/O Observability · GPU · queue metrics · AI/ML Best Practices on EKS Observability GPU · queue metrics Security · isolation · supply chain · AI/ML Best Practices on EKS Security isolation · supply chain Cost Optimization · spot · right-sizing · AI/ML Best Practices on EKS Cost Optimization spot · right-sizing Resource Efficiency · utilization · density · Outcomes to validate Resource Efficiency utilization · density High Performance · latency / goodput / success · Outcomes to validate High Performance latency / goodput / success Reliability · MTTR · incident rate · Outcomes to validate Reliability MTTR · incident rate Cost Savings · measured cost / successful work · Outcomes to validate Cost Savings measured cost / successful work AI/ML Best Practices on EKS Outcomes to validate Legend Best-practice area Expected outcome

Measurement criteria

  • • No fixed speedup or savings percentage is guaranteed.
  • • Include failure, idle time, recovery, transfer and operations.