Fine-tuning Qwen2.5-14B-Instruct on bad medical advice mixed into ordinary chat data makes it broadly misaligned, and deleting the poisoned rows does not undo it. Rewriting a fixed 10% of the poison into correct advice cuts misalignment from 26.8% to 23.1% where deleting the same rows leaves 26.1%, and recovers held-out answer quality that standard benchmarks cannot see. Gradient influence over the rank-1 adapter predicts real delete-and-retrain effects at Spearman 0.87 where the true poison labels score 0.15, and a label-free locate-and-rewrite pipeline ties the labeled rewrite.
How the local learning coefficient scales with width in the lazy and rich learning regimes, set by NTK and muP parametrization. A network linear in its parameters does not look regular under the LLC: its Jacobian rank is capped by the dataset rather than the parameter count, so the NTK network's LLC is nearly width-invariant while the muP network's falls with width, ending about 4x lower at width 1024.
Estimating the LLC means sampling from a posterior held near the trained weights by a spring of stiffness gamma, a number conventionally fixed once and rarely reported. At finite gamma the estimator returns a curvature-weighted count of directions, so fixing gamma across models whose curvature differs by 400x measures each of them differently. Proposes fixing the dimensionless ratio kappa instead and validates the resulting bias against models whose LLC is known exactly.