Huy Nguyen

Huy Nguyen

Ph.D Candidate at the Department of Statistics and Data Sciences, The University of Texas at Austin

Publications

(*) denotes equal contribution, (**) denotes co-last authors

Full Publication List by Themes

Show all publications

I - Statistical Understanding of Modern AI Architectures

MoE Theory

Self-attention Theory

Broader Statistical Foundation

II - Statistical Principles for AI Architecture Design

III- Efficient Adaptation of Large-scale Models

Prompt-based Tuning

Low-rank Adaptation

IV - Optimal Transport

Full Publication List by Year

Show all publications by year

2026

Publications

Preprints

Books

2025

Publications

Preprints

2024

Publications

Preprints

2023

Publications

2022

Publications

2021

Publications

Full Publication List by Type

Show all publications

Journal Submissions

[JS.9] When Does Sparse Routing Help? The Geometry of Expert Separation in Mixture of Experts. arXiv preprint, 2026.
Huy Nguyen, Dung Le, Alessandro Rinaldo, Nhat Ho

[JS.8] On the Geometry of Separation in Finite Gaussian Mixtures. Under review.
Huy Nguyen*, Dung Le*, Alessandro Rinaldo, Nhat Ho

[JS.7] Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport. Under review.
Dung Le*, Huy Nguyen*, Trang Pham, Alessandro Rinaldo, Nhat Ho

[JS.6] On DeepSeekMoE: Statistical Benefits of Shared Experts and Normalized Sigmoid Gating. Under review.
Huy Nguyen, Thong T. Doan, Quang Pham, Nghi D. Q. Bui, Nhat Ho**, Alessandro Rinaldo**

[JS.5] Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspective. Under review
Huy Nguyen*, Fanqi Yan*, Pedram Akbarian, Nhat Ho**, Alessandro Rinaldo**

[JS.4] On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions Under review.
Huy Nguyen*, Xing Han*, Carl William Harris, Suchi Saria**, Nhat Ho**

[JS.3] Joint Estimation and Feature Selection in Overspecified Finite Mixture of Regressions. Under review, 2026
Abbas Khalili*, Pengqi Liu*, Huy Nguyen*, Nhat Ho

[JS.2] Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention. Under review.
Pedram Akbarian*, Huy Nguyen*, Xing Han*, Nhat Ho

[JS.1] On Bayesian Softmax-Gated Mixture-of-Experts Models. Under review.
Nicola Bariletto*, Huy Nguyen*, Nhat Ho**, Alessandro Rinaldo**

Conference Submissions

[CS.6] Sparsity Saves Compute, Not Samples: Oracle Inequalities for Dense and Sparse Mixture of Experts. Under review, 2026
Sagar Ghosh, Huy Nguyen, Alessandro Rinaldo

[CS.5] Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Function. Under review
Tuan Minh Pham*, Thinh Cao*, Viet Nguyen*, Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[CS.4] Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity. Under review
Fanqi Yan*, Dung Le*, Trang Pham, Huy Nguyen, Nhat Ho

[CS.3] Hypernetwork-Driven Low-Rank Adaptation Across Attention Heads. Under review
Nghiem T. Diep*, Dung Le*, Tuan Truong*, Tan Dinh, Huy Nguyen, Nhat Ho

[CS.2] Improving Weight-Decomposed Low-rank Adaptation with Shared Auxiliary Networks. Under review
Nghiem T. Diep*, Hien Dang*, Tuan Truong*, Tan Dinh, Huy Nguyen, Nhat Ho

[CS.1] CompeteSMoE – Statistically Guaranteed Mixture of Experts Training via Competition. Under review.
Nam V. Nguyen, Huy Nguyen, Quang Pham, Van Nguyen, Savitha Ramasamy, Nhat Ho

Journal Publications

[J.2] Convergence Rates for Softmax Gating Mixture of Experts. IEEE Transactions on Information Theory 72(2), 1276-1304, 2026.
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[J.1] EPEM: Efficient Parameter Estimation for Multiple Class Monotone Missing Data. Information Sciences, Volume 567, August 2021, Pages 1-22.
Thu Nguyen, Duy H. M. Nguyen, Huy Nguyen, Binh T. Nguyen, Bruce A. Wade.

Conference Publications

[CS.25] A Statistical Theory of Gated Attention through the Lens of Hierarchical Mixture of Experts. Advances in NeurIPS, 2026
Viet Nguyen*, Tuan Minh Pham*, Thinh Cao*, Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[C.24] One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning. Proceedings of the ICLR, 2026
Minh Le, Bao-Ngoc Dao, Huy Nguyen, Quyen Tran, Anh Nguyen Nhat Ho

[C.23] Revisit Visual Prompt Tuning: The Expressiveness of Prompt Experts. Proceedings of the ICLR, 2026
Minh Le*, Anh Nguyen*, Huy Nguyen, Chau Nguyen, Anh Tran, Nhat Ho

[C.22] On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts. Advances in NeurIPS, 2025
Fanqi Yan*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho**, Alessandro Rinaldo**

[C.21] RepLoRA: Reparameterizing Low-rank Adaptation via the Perspective of Mixture of Experts. Proceedings of the ICML, 2025
Tuan Truong*, Chau Nguyen*, Huy Nguyen*, Minh Le, Trung Le, Nhat Ho

[C.20] On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation. Proceedings of the ICML, 2025
Nghiem T. Diep*, Huy Nguyen*, Chau Nguyen*, Minh Le, Duy M. H. Nguyen, Daniel Sonntag, Mathias Niepert, Nhat Ho

[C.19] Statistical Advantages of Perturbing Cosine Router in Sparse Mixture of Experts. Proceedings of the ICLR, 2025.
Huy Nguyen, Pedram Akbarian*, Trang Pham*, Trang Nguyen*, Shujian Zhang, Nhat Ho

[C.18] Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts. Proceedings of the ICLR, 2025.
Minh Le*, Chau Nguyen*, Huy Nguyen*, Quyen Tran, Trung Le, Nhat Ho

[C.17] Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts. In AISTATS, 2025.
Fanqi Yan*, Huy Nguyen*, Dung Le*, Pedram Akbarian, Nhat Ho

[C.16] Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts. Advances in NeurIPS, 2024.
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[C.15] FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion. Advances in NeurIPS, 2024.
Xing Han, Huy Nguyen*, Carl Harris*, Nhat Ho**, Suchi Saria**

[C.14] Mixture of Experts Meets Prompt-Based Continual Learning. Advances in NeurIPS, 2024.
Minh Le, An Nguyen*, Huy Nguyen*, Trang Nguyen*, Trang Pham*, Linh Van Ngo, Nhat Ho

[C.13] On Least Square Estimation in Softmax Gating Mixture of Experts. Proceedings of the ICML, 2024.
Huy Nguyen, Nhat Ho**, Alessandro Rinaldo**

[C.12] Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts?. Proceedings of the ICML, 2024.
Huy Nguyen, Pedram Akbarian, Nhat Ho

[C.11] A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts. Proceedings of the ICML, 2024.
Huy Nguyen, Pedram Akbarian, TrungTin Nguyen, Nhat Ho

[C.10] Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Experts. Proceedings of the ICLR, 2024.
Huy Nguyen, Pedram Akbarian, Fanqi Yan, Nhat Ho

[C.9] Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts. In AISTATS, 2024.
Huy Nguyen*, TrungTin Nguyen*, Khai Nguyen, Nhat Ho

[C.8] On Parameter Estimation in Deviated Gaussian Mixture of Experts. In AISTATS, 2024.
Huy Nguyen, Khai Nguyen, Nhat Ho

[C.7] Demystifying Softmax Gating Function in Gaussian Mixture of Experts. Advances in NeurIPS, 2023 (Spotlight) .
Huy Nguyen, TrungTin Nguyen, Nhat Ho

[C.6] Minimax Optimal Rate for Parameter Estimation in Multivariate Deviated Models. Advances in NeurIPS, 2023.
Dat Do*, Huy Nguyen*, Khai Nguyen, Nhat Ho

[C.5] Fast Approximation of the Generalized Sliced-Wasserstein Distance. In ICASSP, 2024.
Dung Le*, Huy Nguyen*, Khai Nguyen*, Trang Nguyen*, Nhat Ho

[C.4] Hierarchical Sliced Wasserstein Distance. Proceedings of the ICLR, 2023.
Khai Nguyen, Tongzheng Ren, Huy Nguyen, Litu Rout, Tan Nguyen, Nhat Ho

[C.3] Entropic Gromov-Wasserstein between Gaussian Distributions. Proceedings of the ICML, 2022.
Huy Nguyen*, Khang Le*, Dung Le*, Dat Do, Tung Pham, Nhat Ho

[C.2] On Multimarginal Partial Optimal Transport: Equivalent Forms and Computational Complexity. Proceedings of the AISTATS, 2022.
Huy Nguyen*, Khang Le*, Khai Nguyen, Tung Pham, Nhat Ho

[C.1] On Robust Optimal Transport: Computational Complexity and Barycenter Computation. Advances in NeurIPS, 2021.
Huy Nguyen*, Khang Le*, Quang Minh Nguyen, Tung Pham, Hung Bui, Nhat Ho

Books

[B.1] Handbook of Bayesian Deep Learning.
BayesAI Consortium. CRC Press, 2026 (forthcoming).