Sam Babichenko

Reading notes

Superposition

Networks that represent more sparse features than they have dimensions, and recovering those features.

A Mathematical Framework for Transformer CircuitsElhage et al., 20218 questions Toy Models of SuperpositionElhage et al., 20228 questions Decoding by Linear ProgrammingCandès and Tao, 20058 questions Sparse dictionary learningBricken et al., 20238 questions