Research / Repairing frozen models
CacheMedic++: Robust KV-Cache Stabilization via Self-Distillation
Introduces CacheMedic++, a lightweight in-attention KV-cache repair operator trained by self-distillation on frozen weights, improving gpt2-medium clean score and robustness AUC under held-out cache corruption.
- Published
- 17 February 2026
- Author
- Ali Uyar
Abstract
Large language models rely on the key–value (KV) cache for efficient autoregressive inference, but this persistent internal state is vulnerable to corruption that can induce output drift without changing prompts or weights. We propose CacheMedic++, a lightweight in-attention KV repair operator trained with frozen base model weights via self-distillation. On our canonical gpt2-medium setting, CacheMedic++ improves clean score from 0.2373 to 0.2466 and robustness AUC from 0.0390 (best heuristic) to 0.0401. We further evaluate an explicit contraction regularizer on top of the same distillation recipe: the canonical paired run shows gains (AUC 0.0388 → 0.0401; sensitivity ratios 0.9291 at δ=1.0 and 0.9266 at δ=2.0), but five-seed replication on held-out contiguous_overwrite yields near-neutral average effects (+0.0000 ± 0.0009 AUC gain) with bootstrap intervals spanning no effect. A second multi-seed replication on held-out orthogonal_rotation shows the same pattern (AUC effect −0.0003 ± 0.0003). The same operator family transfers qualitatively to gpt2-large, improving clean score from 0.2974 to 0.3001 and robustness AUC from 0.0472 to 0.0475. Overall, the most reliable signal in this evidence bundle is distillation-driven KV repair; explicit contraction remains inconclusive at current sample size.
Why it matters
The KV cache is persistent internal state, and corrupting it can shift outputs without any change to prompts or weights. Deterministic corruption families make that failure reproducible, so a learned repair can be compared with heuristic ones under held-out corruption types.
Cite
@misc{uyar2026cachemedic,
title = {CacheMedic++: Robust KV-Cache Stabilization via Self-Distillation},
author = {Uyar, Ali},
year = {2026},
doi = {10.5281/zenodo.18669267},
publisher = {Zenodo},
url = {https://doi.org/10.5281/zenodo.18669267}
}