--------------------------------------------------------------------------------- Daniel Manzano Electromagnetism and Condensed Matter Department Institute “Carlos I” for Theoretical and Computational Physics University of Granada Facultad de Ciencias, Av. Fuentenueva s/n Granada 18071, Spain Phone: +34 958241000 Ext: 20569 http://ic1.ugr.es/manzano/ http://ic1.ugr.es/manzano/

Begin forwarded message:

From: Carlos Perez Espigares Subject: [Materia] Charlas de esta semana (8 y 9 de junio a las 11.00) Date: 7. June 2022 at 10:59:44 CEST To: materia@onsager.ugr.es Cc: pietro rotondo , "rcd.gutierrez@gmail.com" , alessia.annibale@kcl.ac.uk

Buenas,

El mes de junio viene cargado de charlas y actividad científica en nuestro grupo aprovechando el final de las clases. Mañana miércoles 8 de junio Pietro Rotondo, de la Universita' degli Studi di Milano hablará sobre "Universal mean field upper bound for the generalisation gap of deep neural networks". Y el jueves 9 de junio Ricardo Gutiérrez, de la Universidad Carlos III de Madrid, nos dará una charla sobre "Dynamical phase transition to localized states in the two-dimensional random walk". No dudéis en difundir estos eventos a quien le pueda interesar.

Saludos, Carlos.

--

Ponente: Dr. Pietro Rotondo. Universita' degli Studi di Milano.

Título: Universal mean field upper bound for the generalisation gap of deep neural networks

Resumen: Modern deep neural networks (DNNs) represent a formidable challenge for theorists: according to the commonly accepted probabilistic framework that describes their performance, these architectures should overfit due to the huge number of parameters to train, but in practice they do not. Here we employ results from replica mean field theory to compute the generalisation gap of machine learning models with quenched features, in the teacher-student scenario and for regression problems with quadratic loss function. Notably, this framework includes the case of DNNs where the last layer is optimised given a specific realisation of the remaining weights. We show how these results – combined with ideas from statistical learning theory – provide a stringent asymptotic upper bound on the generalisation gap of fully trained DNN as a function of the size of the dataset P. In particular, in the limit of large P and Nout (where Nout is the size of the last layer) and Nout ≪ P , the generalisation gap approaches zero faster than 2Nout/P, for any choice of both architecture and teacher function. Notably, this result greatly improves existing bounds from statistical learning theory. We test our predictions on a broad range of architectures, from toy fully-connected neural networks with few hidden layers to state-of-the-art deep convolutional neural networks.

Lugar: Aula de Física Computacional. Facultad de Ciencias.

Fecha y hora: Miércoles 8 de junio. 11:00. _______________________________________________ Materia mailing list Materia@onsager.ugr.es https://onsager.ugr.es/mailman/listinfo/materia