NanoGPT speedrun

The NanoGPT speedrunJordan et al., 2024–20268 questions MuonJordan et al., 2024; Bernstein and Newhouse, 2024; Bernstein, 2025; Amsel et al., 20258 questions Convergence theory for MuonKovalev, 2025; Chen et al., 20258 questions Refinements to MuonLiu et al., 2025; Li et al., 2025; Chen et al., 2025; Srivastava, 20258 questions Learning-rate schedules and batch sizeSchaipp et al., 2025; McCandlish et al., 20188 questions Small-scale proxies for large-scale Transformer training instabilitiesWortsman et al., 20237 questions FlexAttentionDong et al., 2024; records #12-#137 questions Cross-layer connectionsZhou et al., 2024; Xiao et al., 20258 questions Conditional Memory via Scalable LookupCheng et al., 20268 questions