Deepgram moves batch speech recognition to Trainium2 for lower serving costs

Deepgram has ported its Nova-3 speech-to-text model to AWS Trainium2 for high-volume offline transcription, reporting 11% to 70% more audio processed per dollar than the compute options it previously used. The measurement covers throughput-oriented batch jobs such as contact-center analysis and medical transcription.

AWS engineers began with Deepgram’s standard PyTorch implementation, rebuilt it for the Neuron software stack, and tuned kernels and runtime behavior for the accelerator. The final setup runs eight model replicas on a single EC2 trn2.3xlarge instance. Deepgram says its services process quadrillions of audio tokens annually for more than 200,000 developers and 1,400 organizations, so small efficiency gains can materially affect fleet costs.

The reported range depends on workload and instance pricing at the time of evaluation; it is not a universal speedup for live voice agents or every speech model. The work currently targets offline batch inference, where maximizing total audio processed matters more than immediate response time. Customers that self-host Deepgram through SageMaker may also use the optimized path, while still needing to test setup, reliability, and cost against their own traffic.