GTC 2020: Optuna: An Eager Hyperparameter Optimization Library
After clicking “Watch Now” you will be prompted to login or join.
Click “Watch Now” to login or join the NVIDIA Developer Program.
Optuna: An Eager Hyperparameter Optimization Library
Shotaro Sano, Preferred Networks
We'll present Optuna, an open-source, next-generation hyperparameter optimization framework with three novel design criteria: (1) an eager API that allows users to concisely construct dynamic, nested, or conditional search spaces; (2) efficient implementation of both sampling and early-stopping strategies; and (3) an easy-to-set-up, versatile architecture that can be deployed for various purposes, ranging from simple scaling on distributed GPUs to lightweight experimentation, conducted via interactive interface. Optuna is the first optimization software designed to run in Eager mode. We'll present the basic usage of Optuna, describe the design techniques that we needed to meet the above criteria during development, and share our experiences leveraging Optuna to tune hyperparameters on single and multiple GPUs.