AI/ML Infrastructure Layer Architecture

AI/ML Infrastructure Layer Architecture An architecture diagram generated by Archify. Model Training · PyTorch, TensorFlow · ML Workloads Layer Model Training PyTorch, TensorFlow Model Inference · vLLM, TensorRT · ML Workloads Layer Model Inference vLLM, TensorRT Interactive Dev · JupyterHub · ML Workloads Layer Interactive Dev JupyterHub ML Pipelines · Argo Workflows · ML Workloads Layer ML Pipelines Argo Workflows AI Agents · LangChain, CrewAI · ML Workloads Layer AI Agents LangChain, CrewAI Ray Cluster · Distributed compute · Platform Services Layer Ray Cluster Distributed compute KServe · Model serving · Platform Services Layer KServe Model serving Kubeflow · ML platform · Platform Services Layer Kubeflow ML platform MLflow · Experiment tracking · Platform Services Layer MLflow Experiment tracking Vector DB · Milvus, Pinecone · Platform Services Layer Vector DB Milvus, Pinecone GPU NodePools · p4d, p5, g5 · Compute Layer GPU NodePools p4d, p5, g5 Neuron NodePools · inf2, trn1 · Compute Layer Neuron NodePools inf2, trn1 CPU NodePools · m6i, c6i, r6i · Compute Layer CPU NodePools m6i, c6i, r6i Spot Instances · interruption / recovery / cost · Compute Layer Spot Instances interruption / recovery / cost Karpenter · Node provisioning · EKS Base Layer Karpenter Node provisioning EKS Cluster · Control plane · EKS Base Layer EKS Cluster Control plane Storage · EFS, FSx, S3 · EKS Base Layer Storage EFS, FSx, S3 Networking · VPC, EFA · EKS Base Layer Networking VPC, EFA runs on platform Pod placement / execution provisioned by Karpenter ML Workloads Layer Platform Services Layer Compute Layer EKS Base Layer Legend ML workloads Platform services Data stores Compute / base infrastructure

Shared operating boundaries

  • • IAM, data permissions, networking and credentials span every layer.
  • • Tool installation differs from verified model execution and checkpoint recovery.