Physics ∩ ML

a virtual hub at the interface of theoretical physics and deep learning.

16 Jun 2021

Explaining Neural Scaling Laws

Ethan Dyer, Google, 12:00 ET

Abstract: Neural networks exhibit systematic, empirically predictable, performance gains as a function of dataset and model size over many orders of magnitude. I will discuss recent work attempting to understand the origin of these neural scaling laws in deep neural networks and solvable models.