Ray demand, Pod reconciliation, and EC2 capacity

Ray demand, Pod reconciliation, and EC2 capacity An architecture diagram generated by Archify. RayCluster CR · desired worker groups · Architecture component RayCluster CR desired worker groups Ray head Pod · autoscaler sidecar if enabled · Architecture component Ray head Pod autoscaler sidecar if enabled KubeRay operator · reconcile Pods / Services · Architecture component KubeRay operator reconcile Pods / Services Ray worker Pods · CPU / GPU / multi-host groups · Architecture component Ray worker Pods CPU / GPU / multi-host groups Unschedulable Pods · capacity / placement demand · Architecture component Unschedulable Pods capacity / placement demand Karpenter / CA · node provisioning layer · Architecture component Karpenter / CA node provisioning layer EC2 nodes · actual capacity · Architecture component EC2 nodes actual capacity Kubernetes scheduler · Pod placement · Architecture component Kubernetes scheduler Pod placement desired workers watch spec create / reconcile Ray demand no fitting node capacity demand provision nodes capacity place Pods Legend Backend Database Cloud

Separate responsibilities

  • • Ray autoscaler does not send Pods directly to Karpenter
  • • KubeRay reconciles Pods; Kubernetes places them on nodes

Conditions and limits

  • • Check Pending causes; image/PVC/auth failures are not solved by node scaling
  • • Group replicas and Pod counts can differ with numOfHosts