Publications
RL theory monograph: A monograph on RL theory based on notes from courses taught by Nan Jiang at UIUC and together with Sham Kakade at UW. The notes are being actively updated, and any feedback, typos etc. are welcome.Ph.D. Thesis Recent preprints
Journal Publications
Practical Evaluation and Optimization of Contextual Bandit Algorithms On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift Active Learning for Cost-Sensitive Classification
Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization Exact Recovery of Sparsely Used Overcomplete Dictionaries A Reliable Effective Terascale Linear Learning System The Generalization Ability of Online Algorithms for Dependent Data Stochastic convex optimization with bandit feedback Ergodic Mirror Descent Fast global convergence of gradient methods for high-dimensional statistical recovery Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions (Annals formatted version) Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling Message-passing for graph structured linear programs: Proximal projections, convergence and rounding schemes
Conference Publications (see Google Scholar page for most updated version)
Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Sampling Minimax Regret Optimization for Robust Machine Learning under Distribution Shift Adversarially Trained Actor Critic for Offline Reinforcement Learning (Outstanding paper award) Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach Provable RL with Exogenous Distractors via Multistep Inverse Dynamics Bellman-consistent Pessimism for Offline Reinforcement Learning Provably Correct Optimization and Exploration with Non-linear Policies Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation Towards a Dimension-Free Understanding of Adaptive Linear Control PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learning Provably Good Batch Reinforcement Learning Without Great Exploration Policy Improvement from Multiple Experts Safe Reinforcement Learning via Curriculum Induction FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs Taking a hint: How to leverage loss predictors in contextual bandits? On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift On the Optimality of Sparse Model-Based Planning for Markov Decision Processes Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations Off-Policy Policy Gradient with State Distribution Correction Fair Regression: Quantitative Definitions and Reduction-based Algorithms Provably efficient RL with Rich Observations via Latent State Decoding Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches On Polynomial Time PAC Reinforcement Learning with Rich Observations A Reductions Approach to Fair Classification Practical Contextual Bandits with Regression Oracles Hierarchical Imitation and Reinfocement Learning Efficient Contextual Bandits in Non-stationary Worlds Off-policy evaluation for slate recommendation Corralling a Band of Bandit Algorithms Active Learning for Cost-Sensitive Classification Contextual Decision Processes with Low Bellman Rank are PAC-Learnable Optimal and Adaptive Off-policy Evaluation in Contextual Bandits Contextual-MDPs for PAC-Reinforcement Learning with Rich Observations Efficient Second Order Online Learning by Sketching Efficient Contextual Semi-Bandit Learning Fast Convergence of Regularized Learning in Games (Best paper award )Efficient and Parsimonious Agnostic Active Learning Learning to Search Better Than Your Teacher A Lower Bound for the Optimization of Finite Sums Scalable Nonlinear Learning with Adaptive Polynomial Expansions Learning sparsely used overcomplete dictionaries Robust Multi-Objective Learning with Mentor Feedback Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits Least Squares Revisited: Scalable Approaches for Multi-class Prediction Selective sampling algorithms for cost-sensitive multiclass prediction (long version with proofs)Stochastic optimization and sparse statistical recovery: An optimal algorithm for high dimensions (Long version )Contextual Bandit Learning with Predictable Rewards Stochastic convex optimization with bandit feedback Distributed Delayed Stochastic Optimization (Long version )Ergodic Subgradient Descent Learning with Missing Features Oracle inequalities for computationally budgeted model selection (Long version )Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions DIStributed Dual Averaging In Networks Convergence rates of gradient methods for high-dimensional statistical recovery Optimal Algorithms for Online Convex Optimization with Multi-Point Bandit Feedback (longer version with additional proofs)Optimal Allocation Strategies for the Dark Pool Problem Information-theoretic lower bounds on the oracle complexity of convex optimization A Stochastic View of Optimal Regret through Minimax Duality Message-passing for graph structured linear programs: Proximal projections, convergence and rounding schemes An Analysis of Inference with the Universum NIPS 2007 Learning Random Walks to Rank Nodes in Graphs ICML 2007 Learning Parameters in Entity-relationship Graphs from Ranking Preferences with Soumen Chakrabarti ECML/PKDD 2006 Learning to Rank Networked Entities SIGKDD 2006
Teaching
CSE 599: Reinforcement Learning and Bandits , taught at University of Washington in Spring 2019 with Sham Kakade .
Bandits and Reinforcement Learning , taught at Columbia University in Fall 2017 with Alex Slivkins .
Professional Activities
NeurIPS 2022 .
AISTATS 2016.
Optimization for Machine Learning .
Optimization for Machine Learning .
Optimization for Machine Learning .
Optimization for Machine Learning .
Optimization for Machine Learning .
Computational Trade-offs in Statistical Learning .
Learning on Cores, Clusters and Clouds .
Senior Area Chair: NeurIPS 2019, NeurIPS 2020.
Area chair or equivalent: ICML 2013-2020, NeurIPS 2013-2018, COLT 2013-2020, AISTATS 2013, NeurIPS 2013.
Journal Reviewing: JMLR, Annals of Statistics, IEEE Transcations on Automatic Control, IEEE Transcations on Info Theory, SIAM Journal on Optimization, Machine Learning. 
