§ Research Overview

My research focuses on the design and analysis of novel data-driven gradient-based methods to support decision making under uncertainty. Some of my recent interests include:

1. Dynamic optimization: algorithms for Markov decision process (MDP) & reinforcement learning (RL).

2. Robust dynamic optimization: formulation & algorithms for MDPs/RL with uncertain transition kernel/cost.

3. Minimax optimization: optimal gradient-based methods for structured minimax problems.

4. Optimization for machine learning: understanding and improving gradient-based methods in practice.

Part of the above development have also been applied in the context of treatment planning, smart transportation, multi-agent RL, and recommendation systems.

I am constantly interested in applications in broader areas - please kindly reach out if you were interested.

§ Journal

• Distributionally Robust Stochastic Optimal Control
  Alexander Shapiro, Yan Li
  Operations Research Letters, accepted, 2024
  
• Rectangularity and Duality of Distributionally Robust Markov Decision Processes
  Yan Li, Alexander Shapiro
  Mathematical Programming, under review
  
• A Novel Catalyst Scheme for Stochastic Minimax Optimization
  Guanghui Lan, Yan Li
  Mathematical Programming, major revision
  Presented at INFORMS 2023
• First-order Policy Optimization for Robust Policy Evaluation
  Yan Li, Guanghui Lan
  Mathematical Programming, under review
  Presented at MOPTA23
• First-order Policy Optimization for Robust Markov Decision Process
  Yan Li, Guanghui Lan, Tuo Zhao
  Operations Research, major revision
  Presented at MOPTA23, IOS24
• Implicit Regularization of Bregman Proximal Point Algorithm and Mirror Descent on Separable Data
  Yan Li, Caleb Ju, Ethan X. Fang, Tuo Zhao
  Transactions on Machine Learning Research, under review
  
• Policy Mirror Descent Inherently Explores Action Space
  Yan Li, Guanghui Lan
  SIAM Journal on Optimization, accepted, 2024
  Presented at SIAM OP23, IOS24
• Homotopic Policy Mirror Descent: Policy Convergence, Implicit Regularization, and Improved Sample Complexity
  Yan Li, Guanghui Lan, Tuo Zhao
  Mathematical Programming, 2023
  Alice and John Jarvis Ph.D. Student Research Award
  Presented at ICCOPT 2022, INFORMS 2022
• Block Policy Mirror Descent
  Guanghui Lan, Yan Li, Tuo Zhao
  SIAM Journal on Optimization, 2023
  Presented at CISS 2022

§ Conference

• Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms
  Alexander Bukharin, Yan Li, Yue Yu, Qingru Zhang, Zhehui Chen, Simiao Zuo, Chao Zhang, Songan Zhang, Tuo Zhao
  Advances in Neural Information Processing Systems (NeurIPS), 2023
• Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits
  Yan Li, Dhruv Choudhary, Xiaohan Wei, Baichuan Yuan, Bhargav Bhushanam, Tuo Zhao, Guanghui Lan
  International Conference on Learning Representations (ICLR), 2022
• Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably
  Tianyi Liu, Yan Li, Enlu Zhou, Tuo Zhao
  International Conference on Artificial Intelligence and Statistics (AISTAT), 2022
• Pessimism Meets Invariance: Provably Efficient Offline Mean-Field Multi-Agent RL
  Minshuo Chen, Yan Li, Ethan Wang, Zhuoran Yang, Zhaoran Wang, Tuo Zhao
  Advances in Neural Information Processing Systems (NeurIPS), 2021
• Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization
  Tianyi Liu, Yan Li, Song Wei, Enlu Zhou, Tuo Zhao
  International Conference on Artificial Intelligence and Statistics (AISTAT), 2021
• Deep Reinforcement Learning with Robust and Smooth Policy
  Yan Li^*, Qianli Shen^*, Haoming Jiang, Zhaoran Wang, Tuo Zhao
  International Conference on Machine Learning (ICML), 2020
• Implicit Bias of Gradient Descent based Adversarial Training on Separable Data
  Yan Li, Huan Xu, Ethan X. Fang, Tuo Zhao
  International Conference on Learning Representations (ICLR), 2020
• Toward Understanding the Importance of Noise in Training Neural Networks
  Mo Zhou, Tianyi Liu, Yan Li, Dachao Lin, Enlu Zhou, Tuo Zhao
  International Conference on Machine Learning (ICML), 2019
• Non-convex Conditional Gradient Sliding
  Chao Qu, Yan Li, Huan Xu
  International Conference on Machine Learning (ICML), 2018

§ Preprints/Working Papers

• A Markov Decision Process Model for Drivers’ Relocating Behavior in Ride-Hailing Systems
  Anton Kleywegt, Yan Li, Hongzhang Shao
• Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning
  Yan Li, Lingxiao Wang, Jiachen Yang, Ethan Wang, Zhaoran Wang, Tuo Zhao, Hongyuan Zha

§ Research Awards

• Alice and John Jarvis Ph.D. Student Research Award
  Awarded annually to one Ph.D. student (co-winner) in ISyE across all disciplines.
• The Margaret and Stephen Kendrick Research Excellence Award
  Awarded annually to one Ph.D. student (co-winner) in ISyE for research in machine learning and analytics.
• IDEaS-TRIAD Research Scholarship
  Institute-wide scholarship to support research in high-impact cross-disciplinary data science related areas.