Explaining Neural Scaling Laws
Ethan Dyer, Google, 12:00 ETAbstract: Neural networks exhibit systematic, empirically predictable, performance gains as a function of dataset and model size over many orders of magnitude. I will discuss recent work attempting to understand the origin of these neural scaling laws in deep neural networks and solvable models.