AI/ML Workload Deployment

AI/ML Workload Deployment An architecture diagram generated by Archify. NVIDIA GPU Operator · Installed via Helm chart · Amazon EKS NVIDIA GPU Operator Installed via Helm chart NVIDIA Driver · preinstalled in AL2023 AMI · Amazon EKS › GPU layers by component owner NVIDIA Driver preinstalled in AL2023 AMI NVIDIA Container Toolkit · preinstalled in AL2023 AMI · Amazon EKS › GPU layers by component owner NVIDIA Container Toolkit preinstalled in AL2023 AMI NVIDIA Device Plugin · Exposes GPU resources · Amazon EKS › GPU layers by component owner NVIDIA Device Plugin Exposes GPU resources NVIDIA DCGM Exporter · GPU monitoring metrics · Amazon EKS › GPU layers by component owner NVIDIA DCGM Exporter GPU monitoring metrics Kubeflow · ML platform / individual components · Amazon EKS Kubeflow ML platform / individual components Jupyter Notebooks · Interactive development environment · Amazon EKS › Kubeflow Components Jupyter Notebooks Interactive development environment Trainer / legacy Jobs · Runs distributed training jobs · Amazon EKS › Kubeflow Components Trainer / legacy Jobs Runs distributed training jobs Pipelines · Katib · ML workflows · hyperparameter tuning · Amazon EKS › Kubeflow Components Pipelines · Katib ML workflows · hyperparameter tuning KServe · Model serving platform · Amazon EKS KServe Model serving platform Validated serving runtimes · model / device / protocol support · Amazon EKS › Model Serving Options Validated serving runtimes model / device / protocol support MPI Operator · Distributed training via MPIJob · Amazon EKS › Distributed Training MPI Operator Distributed training via MPIJob check ownership provides serving options Amazon EKS GPU layers by component owner Kubeflow Components Model Serving Options Distributed Training Legend Model serving ML platform components GPU node stack

GPU layers

  • • AL2023 supplies driver/toolkit; plugins and observability are separate.
  • • Do not duplicate GPU Operator and standalone plugin ownership.

Training and serving

  • • KServe is the current name; TorchServe is no longer actively maintained.
  • • Resource creation does not prove training/inference success.