Robust Discover→Distill Under Poisoned Distillation Data

Discover→Distill pipelines couple test-time discovery (ephemeral per-task updates) with consolidation (persistent weight updates) to amortize newly discovered behavior. Consolidation is only as trustworthy as the demonstrations it consumes: if archived demonstrations are corrupted or adversarially poisoned, the continual model may internalize incorrect behaviors or backdoors.

We isolate the consolidation stage and study robustness of Self-Distillation Fine-Tuning (SDFT) under poisoned distillation data on a small, fully reproducible synthetic benchmark. We create 4000 distillation pairs from three algorithmic task families (addition, sorting, string reversal) and corrupt a fraction p of demonstrations either (i) random-string corruption or (ii) trigger-target poisoning. We evaluate two low-overhead defenses: weak format gating (D1) and Anchor-KL (D2), a trust-region penalty that keeps the student close to the frozen base model during SDFT.

In a two-seed comparison at one stress point (random corruption, p = 0.3), Anchor-KL had a higher mean clean reward than the no-defense baseline (0.002 270±0.001 443 vs. 0.001 803±0.001 077), but the difference is within seed variability and the ordering is not consistent across the grid: in the single-seed runs Anchor-KL is lower at p = 0.0 and p = 0.1 (random) and in the trigger setting at p = 0.1. By construction, random corruption remains syntactically valid, so format gating drops 0/4000 examples and provides no robustness benefit. In the trigger setting, measured trigger success is 0.0 across all tested conditions; we therefore report no successful backdoor implantation on this benchmark and treat it as a null result.