Regularization and Its Effects on Spurious Feature Learning
Machine Learning Research · PyTorch, ResNet-18, Linear Probes, Interpretability
Studied how L1, L2, and dropout regularization reshape where and when spurious versus core features are encoded across network depth and training time in a ResNet-18. Using linear probes at six layers on Colored MNIST and Waterbirds, we found that each regularization method constrains spurious features through a qualitatively different mechanism.