Quan Xiao
Ph.D. student in
Department of Electrical, Computer, and Systems Engineering
Rensselaer Polytechnic Institute
xiaoq5@rpi.edu
Address: 110 8th Street, Troy, New York, 12180
Bio
I am currently a PhD student at the Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, and am fortunately advised by Prof. Tianyi Chen. Prior to that, I was a visiting student at Data and Computer Science Department of Sun Yat-Sen University during 2020-2021, hosted by Prof. Qing Ling. Previously, I obtained my B.S. degree of Statistics from University of Science and Technology of China in 2020.
At the core of my research is the question: How can optimization shape the next generation of AI? I focus on optimization and machine learning, particularly bilevel and multi-objective optimization, to develop provably efficient algorithms that advance two key frontiers of AI:
- Generative Models: e.g., large language models (LLMs) and diffusion models.
- Next-Generation AI Computing: e.g., emerging analog devices.
News
Feb 2025 Our paper on Analog training for general devices is available. In this work, we demonstrate that Analog SGD has an implicit bias and we analyze the convergence property of Tiki-Taka, a residual learning algorithm for analog training, from the bilevel perspective. Analog In-memory Training on General Non-ideal Resistive Elements: Understanding the Impact of Response Functions .
Feb 2025 Our paper on bilevel optimization for diffusion model is available. In this work, we study two bilevel optimization problems: hyperparameter optimization for fine-tuning diffusion models and noise scheduling for training diffusion models. A First-order Generative Bilevel Optimization Framework for Diffusion Models .
Jan 2025 Our paper Unlocking Global Optimality in Bilevel Optimization: A Pilot Study has been accepted by ICLR 2025! (more info) .
Dec 2024 Our paper On Penalty-based Bilevel Gradient Descent Method has been accepted by Mathematical Programming (Series A)! Final version is coming: On Penalty-based Bilevel Gradient Descent Method .
Dec 2024 I'll attend NeurIPS 2024 and present our work for coupled constrained bilevel optimization at Vancouver, BC. Let's connect! A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints .
Nov 2024 I'll present our recent work on global convergence in bilevel optimization at the Asilomar Conference on Signals, Systems, and Computers and serve as a session co-chair. For a deeper dive, check out the paper: Unlocking Global Optimality in Bilevel Optimization: A Pilot Study .
Oct 2024 Oct 2024: Our paper on optimization foundation for pipeline analog training is now available. Pipeline Gradient-based Model Training on Analog In-memory Accelerators .
Sep 2024 I'm grateful to receive the IEEE Signal Processing Society Scholarship (more info) .
Sep 2024 Our paper on coupled constrained bilevel optimization was accepted by NeurIPS 2024 A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints .
Sep 2024 Our paper on global optimality for bilevel optimization is available. Unlocking Global Optimality in Bilevel Optimization: A Pilot Study .
Aug 2024 Our paper A Bilevel Optimization Method for Inverse Mean-Field Games is accepted by Inverse Problems. (more info) .
June 2024 Our paper on bilevel optimization with coupled constraints is available. A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints .
May 2024 I will join IBM Research as a visiting scholar this summer, under the mentorship of Dr. Debarun Bhattacharjya and managed by Dr. Dharmashankar Subramanian.
Jan 2024 Our paper is available: A Bilevel Optimization Method for Inverse Mean-Field Games .
Dec 2023 I'm grateful to receive the Belsky Award for Computational Sciences and Engineering in RPI (more info) .
Sep 2023 Our paper was accepted by NeurIPS 2023 A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Condition .
Sep 2023 Our paper was accepted by IEEE Transactions on Signal Processing Lazy Queries Can Reduce Variance in Zeroth-order Optimization .
May 2023 I will attend the SIAM Conference on Optimization (OP23) to present our work on bilevel optimization with nonconvex lower-level problem. Check out the paper: A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Condition .
Jan 2023 A short version of our paper on equality-constrained bilevel optimization and its application on federated bilevel learning was accepted by AISTATS 2023 Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization .
May 2022 Our paper was accepted by ICML 2022 Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning .
May 2022 I will join IBM Research as a research extern this summer, under the mentorship of Dr. Songtao Lu and Dr. Lior Horesh
Jan 2022 Our paper was accepted by ICASSP 2022 Federated Multi-armed Bandit via Uncoordinated Exploration .
Jan 2022 Our paper was accepted by AISTATS 2022 (Oral, top 2% of all submissions). A Single-Timescale Method for Stochastic Bilevel Optimization .
Publications
A First-order Generative Bilevel Optimization Framework for Diffusion Models
Quan Xiao, Hui Yuan, A F M Saif, Gaowen Liu, Ramana Kompella, Mengdi Wang, and Tianyi Chen
preprints
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen, Omobayode Fagbohungbe, and Tianyi Chen
preprints
Pipeline Gradient-based Model Training on Analog In-memory Accelerators
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen, Hsinyu Tsai, Kaoutar El Maghraoui, and Tianyi Chen
preprints
Unlocking Global Optimality in Bilevel Optimization: A Pilot Study
Quan Xiao, and Tianyi Chen
Proc. of International Conference on Learning Representations (ICLR), 2025.
A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints
Liuyuan Jiang, Quan Xiao, Victor M. Tenorio, Fernando Real-Rojas, Antonio G. Marques, and Tianyi Chen
Proc. of Neural Information Processing Systems (NeurIPS), 2024.
A Bilevel Optimization Method for Inverse Mean-Field Games
Jiajia Yu, Quan Xiao, Tianyi Chen, and Rongjie Lai
Inverse Problems., Aug. 2024..
A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Condition
Quan Xiao, Songtao Lu, and Tianyi Chen
Proc. of Neural Information Processing Systems (NeurIPS), 2023.
Lazy Queries Can Reduce Variance in Zeroth-order Optimization
Quan Xiao, Qing Ling, and Tianyi Chen
IEEE Transactions on Signal Processing, Sep. 2023.
Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization
Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen
A short version was accepted by Proc. of Intl. Conf. on Artificial Intelligence and Statistics (AISTATS), 2023.
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning
Momin Abbas*, Quan Xiao*, Lisha Chen*, Pin-Yu Chen, and Tianyi Chen (* equal contributor)
Proc. of International Conference on Machine Learning (ICML), 2022.
A Single-Timescale Method for Stochastic Bilevel Optimization
Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin
Proc. of Intl. Conf. on Artificial Intelligence and Statistics (AISTATS), 2022. (Oral, top 2% of all submissions)
A First-order Generative Bilevel Optimization Framework for Diffusion Models
Quan Xiao, Hui Yuan, A F M Saif, Gaowen Liu, Ramana Kompella, Mengdi Wang, and Tianyi Chen
preprints
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen, Omobayode Fagbohungbe, and Tianyi Chen
preprints
Pipeline Gradient-based Model Training on Analog In-memory Accelerators
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen, Hsinyu Tsai, Kaoutar El Maghraoui, and Tianyi Chen
preprints
Unlocking Global Optimality in Bilevel Optimization: A Pilot Study
Quan Xiao, and Tianyi Chen
Proc. of International Conference on Learning Representations (ICLR). 2025.
A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints
Liuyuan Jiang, Quan Xiao, Victor M. Tenorio, Fernando Real-Rojas, Antonio G. Marques, and Tianyi Chen
Proc. of Neural Information Processing Systems (NeurIPS). 2024.
A Bilevel Optimization Method for Inverse Mean-Field Games
Jiajia Yu, Quan Xiao, Tianyi Chen, and Rongjie Lai
Inverse Problems.. Aug. 2024..
A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Condition
Quan Xiao, Songtao Lu, and Tianyi Chen
Proc. of Neural Information Processing Systems (NeurIPS). 2023.
Lazy Queries Can Reduce Variance in Zeroth-order Optimization
Quan Xiao, Qing Ling, and Tianyi Chen
IEEE Transactions on Signal Processing. Sep. 2023.
Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization
Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen
A short version was accepted by Proc. of Intl. Conf. on Artificial Intelligence and Statistics (AISTATS). 2023.
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning
Momin Abbas*, Quan Xiao*, Lisha Chen*, Pin-Yu Chen, and Tianyi Chen (* equal contributor)
Proc. of International Conference on Machine Learning (ICML). 2022.
Federated Multi-armed Bandit via Uncoordinated Exploration
Zirui Yan, Quan Xiao, Tianyi Chen, and Ali Tajer
Proc. of Intl. Conf. on Acoustics, Speech, and Signal Processing (ICASSP). 2022.
A Single-Timescale Method for Stochastic Bilevel Optimization
Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin
Proc. of Intl. Conf. on Artificial Intelligence and Statistics (AISTATS). 2022. (Oral, top 2% of all submissions)
Genome-wide Association Studies of Brain Imaging Data via Weighted Distance Correlations
Canhong Wen, Yuhui Yang, Quan Xiao, Meiyan Huang, and Wenliang Pan
Bioinformatics. 2020.
Image denoising via K-SVD with primal-dual active set algorithm
Quan Xiao, Canhong Wen, and Zirui Yan
Proc. of Winter conference on Applications of Computer Vision (WACV). 2020.
Experience
-
IBM Thomas J. Watson Research Center
- Uncertainty quantification and fine-tuning in LLM. Mentor/Manager: Dr. Debarun Bhattacharjya and managed by Dr. Dharmashankar Subramanian, May-Aug, 2024
- Bilevel optimization with lower-level nonconvexity. Mentor/Manager: Dr. Songtao Lu and Dr. Lior Horesh, May-Aug, 2022
-
School of Data and Computer Science, Sun Yat-Sen University
- Host: Prof. Qing Ling, August 2020 - July 2021
-
Department of Computational Mathematics, Science and Engineering, Michigan State University
- Host: Prof. Yuying Xie, July - October, 2019
- Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov, 2024
- Global Convergence for Bilevel Optimization: A Pilot Study
- Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov, 2023
- Solving A Class of Bilevel Optimization Without Lower-level Strong Convexity
- SIAM Conference on Optimization (OP23), Seattle, WA, June, 2023
- Solving A Class of Bilevel Optimization Without Lower-level Strong Convexity
- Guest Lecturer, RPI ECSE-2500 Engineering Probability, Fall 2024
- Session Co-Chair at the 2024 Asilomar Conference on Signals, Systems, and Computers
- Program Committee Member of AAAI 2025
- Reviewer of AISTATS (2022-2025), NeurIPS (2023 – 2024), ICLR (2024–2025), ICML (2024)
- Reviewer of IEEE Transactions on Signal Processing, IEEE Transactions on Networking
Award
- IEEE Signal Processing Society Scholarship, 2024
- Belsky Award for Computational Sciences and Engineering in Rensselaer Polytechnic Institute, 2023
- NeurIPS 2023 scholar award
- ICML 2022 participation grant
- The excellent student scholarship of the University of Science and Technology of China, 2017-2019