Pinned
Francesco Orabona
2,579 posts
Francesco Orabona
@bremen79
Dad and associate professor at @KAUST_News.
Formerly @BU_ece, @sbucompsc, @YahooResearch, @TTIC_Connect.
ML theory&practice, obsessed with history of science
For the future PhD students that are deciding in these days which offer to accept, let me give you a very simple advice: Avoid choosing a**holes as supervisors. It does not matter how famous they are or how prestigious is the school, you'll regret it and they don't deserve you.- New blog post: Neural Networks (Maybe) Evolved to Make ADAM the Best Optimizer For the first time, I wrote a blog post without math :) I discuss a *conjecture* I have regarding Adam and the way the deep learning community produces new ideas
- As promised, I compiled all my lecture notes on Online Learning in a single PDF. Feedback is welcome! "A Modern Introduction to Online Learning" arxiv.org/abs/1912.13213 PS Happy New Year!
- You cannot take the logarithm of the Lipschitz constant of a function! A 🧵 about a super common mistake in ML papers 1/10
- It is now official: I got the @NSF CAREER award for "Parameter-free Optimization Algorithms for Machine Learning"! Super happy :) Soon, I'll tweet about an exciting new result about parameter-free algorithms for non-convex functions: I had this idea writing this proposal :)
- This is a turning point: I just proved a complex math result useful for my research using an LLM. I am not sure if I should be happy or scared...
- New version of my book on Online Learning! arxiv.org/abs/1912.13213 This is a BIG update, with 43 pages of additional content. Let me list some of the changes. 🧵1/8
- Dear PhD students with a submission to @NeurIPSConf, soon ~20% of you will receive a desk reject. Here some suggestions to deal with it in a healthy way. First, this is not personal: a paper written by you was rejected, not you. Keep your self-worth unlinked from your work. 1/5
- New blog post: Yet Another ICML Award Fiasco The story of the @icmlconf 2023 Outstanding Paper Award to the D-Adaptation paper with worse results that the ones from 9 years ago Please share it to start a needed conversation on mistakenly granted awards
- As promised, we put on Arxiv the proof we did with Gemini. arxiv.org/pdf/2505.20219 This shows that the Polyak stepsize not only will not reach the optimum, but it can cycle, when used without the knowledge of f*. Gemini failed when prompted directly ("Find an example where the
This is a turning point: I just proved a complex math result useful for my research using an LLM. I am not sure if I should be happy or scared... - Unpopular opinion: "Optimization for Deep Learning" should be removed as a subtopic in ML conferences. Numerical optimization is estabilished field: we don't need a worse variant of it with algorithms justified by "intuition", rarely converging, and not properly evaluated.
- Given that even @BU_ece changed my title on the website, I guess it is now official: I have been awarded tenure and promoted to Associate Professor This was a long and stressful journey, but I f*cking did it!!! 💪🥳🎉
