AssemblyAI says it reduced speech-to-text inference costs by 70% by running a diversified mix of Amazon EC2 Spot and on-demand GPU instances. Its architecture accepts that AWS can reclaim Spot capacity with two minutes’ notice and moves interrupted jobs back into a queue for another worker.
The platform runs on Amazon ECS and uses Nvidia GPU instances from the g4dn, g5 and g6 families across multiple availability zones. On-demand capacity provides an always-on baseline, while Spot instances absorb additional work. Spreading demand across hardware families increases the available capacity pool and reduces reliance on any single market.
Amazon SQS separates job submission from execution. When an interruption notice arrives, work that cannot finish within two minutes returns to the main queue, where another healthy instance can continue it. Repeated failures go to a dead-letter queue for investigation. Custom machine images containing model artifacts and containers can start in about two minutes, limiting recovery time.
AssemblyAI also deploys regionally, including London, to meet European data-residency needs. The 70% figure is a company-reported result from this workload, not a universal Spot discount. Teams considering the pattern need retry-safe jobs, monitoring and enough on-demand capacity to maintain their own latency and availability targets when cheap GPU capacity disappears.