IBM Research unveiled a “Distributed Deep Learning” (DDL) library that enables cuDNN-accelerated deep learning frameworks like TensorFlow, Caffe, Torch and Chainer to scale to tens of IBM servers leveraging hundreds of GPUs.
“With the DDL library, it took us just 7 hours to train ImageNet-22K using ResNet-101 on 64 IBM Power Systems servers that have a total of 256 NVIDIA P100 GPU accelerators in them,” mentioned Sumit Gupta, VP, HPC, AI & Machine Learning at IBM Cognitive Systems. “16 days down to 7 hours changes the workflow of data scientists. That’s a 58x speedup!”
According to the researcher’s paper, the team achieved deep learning records in image recognition accuracy and training times when using the new library and 256 GPUs.
A technical preview of DDL is available in version 4 of IBM’s PowerAI enterprise deep learning software, which makes this cluster scaling feature available to any organization using deep learning for training their AI models.
Read more >
AI-Generated Summary
- IBM Research unveiled a Distributed Deep Learning (DDL) library that enables cuDNN-accelerated deep learning frameworks like TensorFlow, Caffe, Torch and Chainer to scale to tens of IBM servers leveraging hundreds of GPUs.
- Using the DDL library with 256 NVIDIA P100 GPU accelerators on 64 IBM Power Systems servers, the team trained ImageNet-22K with ResNet-101 in 7 hours, achieving a 58x speedup over the previous 16-day baseline.
- The researchers' paper reports deep learning records in image recognition accuracy and training times when using the new library and 256 GPUs.
Next Steps
- Read the IBM Research blog post for additional details on the Distributed Deep Learning library.
- Explore PowerAI enterprise deep learning version 4 to evaluate the technical preview of the DDL library.
Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more
About the Authors
