Apple researchers have proposed an adaptive way to steer generative models only when their output shows signs of unwanted behavior. Dynamically Scaled Activation Steering, or DSAS, separates the decision about when to intervene from the transformation used to change a model’s internal activity.
Conventional activation steering applies a learned direction across inputs, which can suppress toxicity or encourage a trait but may also damage useful behavior when no intervention is needed. DSAS instead calculates context-dependent scaling factors during generation. It can apply a stronger correction to layers and tokens associated with the targeted behavior while leaving other parts of the response largely untouched. The framework can sit on top of existing steering methods or be optimized jointly with one.
In the researchers’ tests, adding DSAS improved the trade-off between toxicity reduction and preserving model utility compared with steering alone. They also applied the method to a text-to-image diffusion model, where it adaptively changed specific concepts. The team says the approach adds little computational overhead and makes interventions easier to inspect by showing which tokens were steered and by how much. These are research results rather than a product release, and the project page says code will be made available on GitHub.