Ternary language models do not use their three possible weight values equally, allowing a new storage layout to beat the familiar 1.58-bit theoretical figure in practice. Researchers measured 29 models and found that zero-valued weights made up as much as 51.5% of their parameters.
Conventional five-trit packing stores five values from {-1, 0, +1} in one byte, which works out to 1.625 bits per weight with common power-of-two group sizes. BITCOS instead records whether each weight is present in a dense bitmap and stores a compact sign vector only for nonzero values. Its cost is `2 - z` bits per weight, where `z` is the fraction of zeros. The method used less space than five-trit packing in 26 of the 29 tested models and reached 1.485 bits per weight on the sparsest.
The layout is designed for direct unpacking on AVX-512, AVX2 and Intel Xe2 hardware rather than archival compression alone. Against existing ternary matrix-vector multiplication kernels, it delivered gains up to 1.28 times at observed zero densities. End-to-end decoding improved by as much as 1.18 times on CPUs and 1.27 times on GPUs across five client and server platforms. Benefits depend on a model’s zero distribution and hardware implementation, so dense ternary models may see little advantage.