I would also like to shout-out @bremen79 @xyli94 @CevherLIONS (among many others) for their inspiring works and progress in this area!
Check out our #COLT2022 paper extending the analysis of AdaGrad for nonconvex smooth functions to settings where gradients can be unbounded and variance of stochastic gradients scales affinely. In this thread, I describe our new technical tools needed to achieve this result.
1/9
