Accelerating KubeFlow Pipeline with NVIDIA RAPIDS and GPUs on Kubernetes

Discuss (0)

Data science workflows are inherently complex. They scale across clusters of servers running software from different parts of the workflow, and they are often compute-intensive. All this results in slow machine learning model development and deployment cycles.
To help speed up end-to-end data science training, NVIDIA developed RAPIDS, an open-source data analytics and machine learning acceleration platform designed exclusively for GPUs. As a step towards wider integration, the KubeFlow team announced the availability of the NVIDIA RAPIDS GPU-accelerated libraries as an image on the Kubeflow Pipelines.

What is KubeFlow

KubeFlow is a cloud-native platform for machine learning built on top of Kubernetes. The platform reduces machine learning production by providing end-to-end workflows in an environment that can be efficiently scaled to production through automation. Kubeflow also integrates a collection of Google-developed frameworks that enable data scientists and ML developers to focus on pipeline optimization.
“The integration of RAPIDS with KubeFlow Pipelines streamlines the model development workflow and drastically decreases end-to-end model iterations times by automating the deployment of open GPU-accelerated data science tools,” the KubeFlow team wrote in a blog post. “Combining the simple orchestration of machine learning pipelines with RAPIDS, a collection of CUDA-accelerated libraries, data scientists can train and deploy machine learning pipelines significantly faster to solve business problems.”

Read more