Sam Babichenko

Reading notes

High-dimensional SGD

How many samples SGD needs to find a hidden direction in high dimension.

The information exponentBen Arous et al., 2021; Damian et al., 20238 questions Feature learning from one gradient stepBa et al., 2022; Damian et al., 20228 questions The merged-staircase propertyAbbe et al., 20228 questions Learning sparse parities with SGDBarak et al., 20228 questions