Researchers behind JustFit report running Qwen3.8-27B with a completed context of 212,992 token positions on a 24GB M4 Pro MacBook. That is 6.93 times the 30,720-position capacity they measured with the mlx-vlm baseline, showing how inference-state management can expand local model context without changing the model’s weight quantization.
The MLX-based runtime combines three mechanisms. KVExec compresses and reconstructs the key-value cache used to remember earlier tokens. PhaseSwap moves components in and out of memory according to the current inference stage. StateTrans preserves model state while the serving configuration changes. In three capacity runs, the system completed 196,608 input tokens followed by 16,384 output tokens; a separate two-request test retained 229,376 positions in total.
Performance remains part of the trade-off. A test with 32,000 input tokens and 64 output tokens reached 19.11 generated tokens per second, while a repeated 32K-plus-6K workload peaked at a median 16,374 MiB of process memory. The integrated setup answered 29 of 30 AIME 2026 math problems correctly. These are results from the authors’ specific laptop, model and runtime rather than broad production benchmarks, but they demonstrate that careful state movement can make long-context local inference practical on hardware far smaller than a server GPU.