Topic: regularization
-
Early stopping: patience, validation and restoring the best weights
Treat the stopping epoch as a model choice, monitor a separate validation signal, and distinguish the last trained weights from the best observed weights.
-
AdamW: why decoupled weight decay differs from an L2 penalty
AdamW shrinks parameters separately from the adaptive gradient update. This prevents the decay term entering Adam’s moment estimates, but learning rate, update count and parameter-group choices still affect regularization.
Machine-readable: JSON