Ray on EKS Deep Dive
Supported Versions: Ray 2.57.0, KubeRay v1.6.1 Last Updated: August 20, 2026
Overview
Ray is an open-source distributed computing framework for scaling Python workloads — from ad hoc parallel tasks to distributed training, hyperparameter tuning, and model serving — built around a small set of core primitives (tasks, actors, and a shared object store) rather than a separate tool per workload type. On Kubernetes, the KubeRay operator translates a Ray cluster's head/worker-node shape into native Kubernetes resources, making Ray clusters declarative and giving EKS the same deployment and autoscaling story it already uses for other workloads.
Component Map
| Concept | Problem It Solves | Deep Dive |
|---|---|---|
| Architecture | Tasks, actors, and the object store that everything else builds on | Part 1 |
| KubeRay Operator | Run Ray clusters as native Kubernetes resources (RayCluster/RayJob/RayService) | Part 2 |
| Ray Train & Tune | Distributed model training and hyperparameter search | Part 3 |
| Ray Serve | Model serving, including dedicated LLM-serving building blocks | Part 4 |
Why Run This on EKS
The trade-off is the same one covered elsewhere in this documentation site's data/ML sections: a team already running EKS can reuse the same node-pool autoscaling (via Karpenter), IAM, and observability patterns for Ray workloads as for everything else on the cluster, in exchange for operating the KubeRay operator and its RayCluster/RayJob/RayService resources directly rather than using a managed alternative.
Currently Covered
- Part 1: Ray Architecture — tasks, actors, the object store, and the head/worker cluster model
- Part 2: The KubeRay Operator — RayCluster, RayJob, RayService, and the two-tier autoscaling pattern with Karpenter
- Part 3: Ray Train and Ray Tune — distributed training and hyperparameter tuning
- Part 4: Ray Serve — model serving, Ray Serve LLM, and RayService-based production deployment