Ai2 has published a Hugging Face post on DiScoFormer, a transformer architecture designed to handle both density and score modeling across distributions. The work targets a technical problem in generative modeling: how to represent related distributional information in one system.
The approach matters because score-based and density-based methods are often treated as separate modeling paths. A unified transformer could simplify experimentation and make some generative-model workflows easier to compare.
For researchers, the post is another sign that transformer architectures continue to expand beyond text into broader probabilistic modeling tasks.