Latency is how long an AI API request takes. It includes network time, queue time, model processing time, tool calls, retrieval, and output generation.
In practice
Latency matters for chatbots, live support, coding assistants, search, and voice interfaces. Faster models, shorter prompts, streaming, caching, and simpler workflows can improve the user experience.
What to watch
The fastest model is not always the best choice. Balance latency with quality, cost, and reliability for the actual workflow.