AWS publishes deployment paths for Nvidia’s 120B multimodal Nemotron model

AWS has published deployment guidance for Nvidia Nemotron 3.5 Super VL on Amazon EKS, EKS Auto Mode and SageMaker HyperPod EKS. The open multimodal model accepts text, images, documents and video, then produces text for tasks such as document analysis, coding and security triage.

Nemotron uses a mixture-of-experts design with 120 billion total parameters and 12 billion active for each token. Its architecture combines Mamba-2, attention and expert layers with a vision encoder, and supports a context window of up to one million tokens. Nvidia publishes weights, training datasets and post-training recipes for customization.

Sparse activation reduces computation per token, but it does not make the full model small. AWS lists minimum vLLM configurations ranging from one B300 or GB300 GPU for an NVFP4 checkpoint to 16 H100 GPUs for BF16. The hardware still has to hold the model weights even when only a fraction of experts runs at each step.

Nvidia reports benchmark results for agent and chart-reasoning tasks, but they are model-maker figures and do not establish performance for a specific workload. Teams considering self-hosting should compare quality, latency and utilization against smaller models or managed APIs; the deployment recipe solves infrastructure assembly, not the economics of keeping expensive accelerators busy.