Maths with transformers
François Charton, MetaAbstract:
Transformers can be trained, from synthetic data, to solve problems of mathematics, by considering them as translations from problems into solutions. Models achieve high accuracy on a variety of tasks, learn deep mathematical properties, and generalize out of distribution if their training set is selected with care. I illustrate their use on several problems, from symbolic integration to numerical computation, and symbolic regression.