Hemil Desai

Hemil Desai is a senior software engineer at NVIDIA, where he focuses on building scalable, high-performance infrastructure for generative AI and large-scale models. His technology interests span deep learning systems, distributed training frameworks, and optimizing PyTorch workloads at GPU-scale. Hemil holds a master’s in computer science from University of California, Los Angeles and a bachelor’s in computer science from Purdue University.
Avatar photo

Posts by Hemil Desai

Agentic AI / Generative AI

Accelerating Large-Scale Mixture-of-Experts Training in PyTorch

Training massive mixture-of-experts (MoE) models has long been the domain of a few advanced users with deep infrastructure and distributed-systems expertise.... 7 MIN READ