Here’s my take on the “mathematical foundations” of machine learning and AI. These course notes cover the basics of statistical learning theory, optimization, and functional analysis. nowak.ece.wisc.edu/MFML.pdf
My dad (86 years old today!) is really into fractals and art. He’s sitting in on my ML course on neural nets etc, and he decided to start experimenting with fractal images generated using ReLU nonlinearities.
What kinds of functions do deep neural networks learn? Now we know!
arxiv.org/abs/2105.03361
Our paper presents a new representer theorem for deep ReLU networks and provides theoretical insights into weight decay, sparsity, skip connections, and low-rank weight matrices.
Rahul Parhi and I wrote a tutorial style article to explain modern theory of deep learning and neural function spaces via elementary signal/image processing concepts (Fourier transform, Radon transform, L1 regularization, sparsity) arxiv.org/abs/2301.09554
What kinds of functions do neural networks learn? I gave a lecture on the topic at the Joint Statistical Meetings (JSM) this week. Slides are online here:
I am immensely honored to be named the Grace Wahba Professor of Data Science. Grace has been a tremendous inspiration and guiding force throughout my entire career. research.wisc.edu/faculty-receiv…
We are pleased to announce plans for a special issue of Signal Processing Magazine focused on the mathematics of deep learning:
signalprocessingsociety.org/blog/ieee-spm-…
We look forward to your submissions!
Congratulations to @yinglun122, who successfully defended his PhD thesis "Interactive ML: From Theory to Scale". He'll be starting at UC-Riverside as an Assistant Prof this summer!
