Laplace 3B keeps serving memory fixed as prompts grow

A 3-billion-parameter model from i14 keeps its per-sequence serving state fixed at 29.5 MB regardless of prompt length. Laplace 3B replaces standard transformer attention with gated linear attention, compressing earlier context into a recurrent matrix instead of retaining an expanding key-value cache.

Working with AWS, i14 ported the custom architecture from GPU-oriented Triton code to native PyTorch on Trainium2. A distributed optimizer change reduced communication per training step from 10.87 GB to 0.45 GB per rank and raised reported throughput 1.63 times.

On one trn2.48xlarge instance, training settled near 120,000 tokens per second. The model served 11,544 output tokens per second across 1,280 concurrent sequences through an unmodified vLLM engine. Its recurrent state was roughly 82 times smaller than a comparable cache at 8,192 tokens.

Fixed memory can improve predictability for long-running agents and large documents, but compression introduces a tradeoff: the model does not preserve every prior key and value. Teams still need to test whether it retrieves the right old information for their workload, not just whether it handles longer prompts cheaply.