Research / Inside the model
Surgical Post-Training Diffing: Partial Recovery Without Clean Small-Mask Separation
Builds a paired answer-phase sparse surrogate that captures part of instruction tuning in Gemma 3 4B, cutting answer-token KL to the instruction-tuned model by 28.3%.
- Published
- 31 March 2026
- Author
- Ali Uyar
Abstract
Instruction-tuned models differ from their pretrained siblings in ways that are behaviorally useful but internally hard to localize. We study this gap with a paired answer-phase sparse surrogate built from two residual-stream layers under a shared neutral template. On Gemma 3 4B PT/IT, the learned surrogate lowers answer-token KL to the instruction-tuned model by 28.3%, raises a held-out capability composite from 0 to 0.070 (0.186 recovery of the PT-to-IT gap), and sharply reduces the pretrained model’s long repeated continuations. However, the more ambitious separation story fails on held-out data. A frozen 5-feature capability mask recovers no capability at all, matching trivial baselines, and subtracting a learned verbosity mask preserves recovered capability but lengthens outputs from 16.65 to 17.84 tokens on average even as the carryover-based verbosity score falls from 0.923 to 0.886. A late-only one-layer ablation nearly matches the two-layer surrogate, and a threshold sweep shows the null capability-mask result is robust to nearby selector cutoffs. The resulting picture is more precise than a simple positive or negative headline: a small answer-phase surrogate can capture a real part of instruction-tuning behavior, but tiny frozen masks do not cleanly split capability from verbosity in this compact setting.
Why it matters
Instruction tuning changes behavior in ways that are hard to localize. A paired sparse surrogate gives part of that change an explicit form, and held-out tests measure how much it recovers and whether small feature masks can split capability from verbosity.
Cite
@misc{uyar2026surgical,
title = {Surgical Post-Training Diffing: Partial Recovery Without Clean Small-Mask Separation},
author = {Uyar, Ali},
year = {2026},
doi = {10.5281/zenodo.19347476},
publisher = {Zenodo},
url = {https://doi.org/10.5281/zenodo.19347476}
}