AWS has added managed Ray capabilities to SageMaker HyperPod, reducing the Kubernetes setup needed to run distributed training and inference. Data scientists can now create a Ray cluster, connect JupyterLab or Code Editor, submit jobs and open monitoring dashboards from SageMaker Studio.
Ray is an open-source framework that distributes Python workloads across CPU and GPU clusters. Previously, teams running it on Kubernetes often had to write manifests, rebuild container images for dependency changes, configure port forwarding and assemble Prometheus and Grafana monitoring themselves.
The HyperPod integration uses the open-source KubeRay operator and standard Ray APIs, so existing scripts do not require a proprietary rewrite. It adds node-health monitoring, automatic recovery, hung-job detection and tiered checkpoints for faster restarts. SageMaker JumpStart can also load model weights into Ray Serve endpoints, with key-value cache offloading intended for long-context inference.
Customers need a SageMaker HyperPod cluster orchestrated by Amazon EKS plus a SageMaker Studio domain and the required platform components. Advanced users can still edit the underlying Kubernetes manifest, while administrators can apply compute quotas and scheduling priorities through HyperPod task governance.