AWS has released Ray Serve Deep Learning Containers for model inference as TorchServe reaches the end of active maintenance. TorchServe no longer has planned feature, compatibility or security updates, leaving users responsible for keeping PyTorch, CUDA and serving dependencies aligned.
The AWS images package PyTorch, Ray Serve, FastAPI, Uvicorn and common media utilities with a tested operating-system and GPU stack. Separate entry points support Amazon EKS and EC2 as well as SageMaker. AWS says it applies security patches during image builds, while developers can layer extra libraries onto the base when a model needs them.
A reference deployment serves the Qwen3-VL-2B vision-language model on one EKS node with an Nvidia A10G GPU and 24 GB of memory. The application code arrives through a Kubernetes ConfigMap, avoiding a custom image rebuild for each serving change. That example is a single-node starting point, not a turnkey migration for large distributed deployments; teams need KubeRay or another scaling design for multiple workers and replicas.