AWS has published a reference architecture for placing LiteLLM in front of Amazon Bedrock as an OpenAI-compatible gateway. Developers configure coding assistants with one base URL and personal key, while platform teams centrally manage model routes, budgets, quotas and observability.
The design runs LiteLLM on Fargate behind a load balancer. PostgreSQL stores key and spending state, Valkey coordinates caching and rate limits, and Bedrock application inference profiles isolate model quotas. CloudWatch covers infrastructure metrics while Langfuse can capture prompt-level traces. Teams must decide whether sensitive prompts and responses should be logged and apply suitable redaction and retention.
Two configuration details can silently break controls. When a raw Bedrock inference-profile ARN is used, per-token prices must be set explicitly or LiteLLM may record zero cost and never enforce budgets. Global cross-region inference also requires permission for the underlying foundation model in every possible destination region, not only the application-profile ARN.
The post says AWS tested the pattern with several hundred developers, but its code snippets are illustrative rather than a one-command deployment. Self-service key issuance requires LiteLLM Enterprise beyond a small free pilot. Fargate, PostgreSQL, Valkey, the load balancer and related services continue billing until removed, so teams should evaluate the operational cost against simpler direct access.