Research Overview

I am interested in the computational perspective of learning optimal decisions from data. Specifically I tend to focus on the design and analysis of first-order methods and their applications in data science. Some of my recent interests include:

• Dynamic optimization: Design computation- and sample-efficient algorithms for Markov decision process (MDP) and stochastic optimal control (SOC).
• Distributionally robust/risk-averse dynamic optimization: Develop reasonable formulations and scalable algorithms for MDP and SOC with distributional ambiguity.
• Minimax optimization: Design gradient-based methods for minimax problems and dynamic games.
• Optimization for machine learning: Understand and improve gradient-based methods in ML practice.

You may also find projects I have done (some still ongoing) in collaboration with industry here. They cover applications from treatment planning, urban transportation control, and recommendation systems.

Journal

• Distributionally Robust Stochastic Optimal Control
  Alexander Shapiro, Yan Li
  Operations Research Letters, accepted, 2024

• Rectangularity and Duality of Distributionally Robust Markov Decision Processes
  Yan Li, Alexander Shapiro
  Mathematical Programming, under review

• A Novel Catalyst Scheme for Stochastic Minimax Optimization
  Guanghui Lan, Yan Li
  Mathematical Programming, major revision

• First-order Policy Optimization for Robust Policy Evaluation
  Yan Li, Guanghui Lan
  Mathematical Programming, under review

• First-order Policy Optimization for Robust Markov Decision Process
  Yan Li, Guanghui Lan, Tuo Zhao
  Operations Research, major revision

• Implicit Regularization of Bregman Proximal Point Algorithm and Mirror Descent on Separable Data
  Yan Li, Caleb Ju, Ethan X. Fang, Tuo Zhao
  Transactions on Machine Learning Research, under review

• Policy Mirror Descent Inherently Explores Action Space
  Yan Li, Guanghui Lan
  SIAM Journal on Optimization, accepted, 2024

• Homotopic Policy Mirror Descent: Policy Convergence, Implicit Regularization, and Improved Sample Complexity
  Yan Li, Guanghui Lan, Tuo Zhao
  Mathematical Programming, 2023
  Alice and John Jarvis Ph.D. Student Research Award

• Block Policy Mirror Descent
  Guanghui Lan, Yan Li, Tuo Zhao
  SIAM Journal on Optimization, 2023

Conference

• Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms
  Alexander Bukharin, Yan Li, Yue Yu, Qingru Zhang, Zhehui Chen, Simiao Zuo, Chao Zhang, Songan Zhang, Tuo Zhao
  Advances in Neural Information Processing Systems (NeurIPS), 2023

• Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits
  Yan Li, Dhruv Choudhary, Xiaohan Wei, Baichuan Yuan, Bhargav Bhushanam, Tuo Zhao, Guanghui Lan
  International Conference on Learning Representations (ICLR), 2022

• Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably
  Tianyi Liu, Yan Li, Enlu Zhou, Tuo Zhao
  International Conference on Artificial Intelligence and Statistics (AISTAT), 2022

• Pessimism Meets Invariance: Provably Efficient Offline Mean-Field Multi-Agent RL
  Minshuo Chen, Yan Li, Ethan Wang, Zhuoran Yang, Zhaoran Wang, Tuo Zhao
  Advances in Neural Information Processing Systems (NeurIPS), 2021

• Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization
  Tianyi Liu, Yan Li, Song Wei, Enlu Zhou, Tuo Zhao
  International Conference on Artificial Intelligence and Statistics (AISTAT), 2021

• Deep Reinforcement Learning with Robust and Smooth Policy
  Yan Li^*, Qianli Shen^*, Haoming Jiang, Zhaoran Wang, Tuo Zhao
  International Conference on Machine Learning (ICML), 2020

• Implicit Bias of Gradient Descent based Adversarial Training on Separable Data
  Yan Li, Huan Xu, Ethan X. Fang, Tuo Zhao
  International Conference on Learning Representations (ICLR), 2020

• Toward Understanding the Importance of Noise in Training Neural Networks
  Mo Zhou, Tianyi Liu, Yan Li, Dachao Lin, Enlu Zhou, Tuo Zhao
  International Conference on Machine Learning (ICML), 2019

• Non-convex Conditional Gradient Sliding
  Chao Qu, Yan Li, Huan Xu
  International Conference on Machine Learning (ICML), 2018

Preprints/Working Papers

• Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning
  Yan Li, Lingxiao Wang, Jiachen Yang, Ethan Wang, Zhaoran Wang, Tuo Zhao, Hongyuan Zha

Research Awards

Part of my research has been kindly acknowledged by

• Alice and John Jarvis Ph.D. Student Research Award
  Awarded annually to one Ph.D. student (co-winner) in ISyE across all disciplines.

• The Margaret and Stephen Kendrick Research Excellence Award
  Awarded annually to one Ph.D. student (co-winner) in ISyE for research in machine learning and analytics.

• IDEaS-TRIAD Research Scholarship
  Institute-wide scholarship to support research in high-impact cross-disciplinary data science related areas.