How the local learning coefficient scales with width in the lazy and rich learning regimes, set by NTK and muP parametrization. A network linear in its parameters does not look regular under the LLC: its Jacobian rank is capped by the dataset rather than the parameter count, so the NTK network's LLC is nearly width-invariant while the muP network's falls with width, ending about 4x lower at width 1024.
Estimating the LLC means sampling from a posterior held near the trained weights by a spring of stiffness gamma, a number conventionally fixed once and rarely reported. At finite gamma the estimator returns a curvature-weighted count of directions, so fixing gamma across models whose curvature differs by 400x measures each of them differently. Proposes fixing the dimensionless ratio kappa instead and validates the resulting bias against models whose LLC is known exactly.