Papers

Rethinking On-Policy Self-Distillation for Thinking Models
Simran Kaur, Narutatsu Ri, Yinghui He, Liam Fowl, Sanjeev Arora
arXiv
Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
Yinghui He, Simran Kaur, Adithya Bhaskar, Yongjin Yang, Jiarui Liu, Narutatsu Ri, Liam Fowl, Abhishek Panigrahi, Danqi Chen, Sanjeev Arora
arXiv
How Does RL Post-training Induce Skill Composition? A Case Study on Countdown
Simon Park*, Simran Kaur*, Anirudh Goyal, Sanjeev Arora
arXiv
Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning
Simran Kaur*, Simon Park*, Anirudh Goyal, Sanjeev Arora
arXiv
Can Models Learn Skill Composition from Examples?
Haoyu Zhao, Simran Kaur, Dingli Yu, Anirudh Goyal, Sanjeev Arora
arXiv
Skill-Mix: a Flexible and Expandable Family of Evaluations for AI models
Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal, Sanjeev Arora
arXiv demo PLI blog post Quanta Magazine Article
Disentangling the Mechanisms Behind Implicit Regularization in SGD
Zachary Novack, Simran Kaur, Tanya Marwah, Saurabh Garg, Zachary Lipton
arXiv
On the Maximum Hessian Eigenvalue and Generalization
Simran Kaur, Jeremy Cohen, Zachary Lipton
arXiv
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
We empirically demonstrate that full-batch gradient descent on neural network training objectives typically operates in a regime we …
Jeremy Cohen, Simran Kaur, Yuanzhi Li, Zico Kolter, Ameet Talwalker
arXiv
Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?
Simran Kaur, Jeremy Cohen, Zachary Lipton
arXiv