Data science workflows are inherently complex. They scale across clusters of servers running software from different parts of the workflow, and they are often compute-intensive. All this results in slow machine learning model development and deployment cycles.
To help speed up end-to-end data science training, NVIDIA developed RAPIDS, an open-source data analytics and machine learning acceleration platform designed exclusively for GPUs. As a step towards wider integration, the KubeFlow team announced the availability of the NVIDIA RAPIDS GPU-accelerated libraries as an image on the Kubeflow Pipelines.
What is KubeFlow
KubeFlow is a cloud-native platform for machine learning built on top of Kubernetes. The platform reduces machine learning production by providing end-to-end workflows in an environment that can be efficiently scaled to production through automation. Kubeflow also integrates a collection of Google-developed frameworks that enable data scientists and ML developers to focus on pipeline optimization.
“The integration of RAPIDS with KubeFlow Pipelines streamlines the model development workflow and drastically decreases end-to-end model iterations times by automating the deployment of open GPU-accelerated data science tools,” the KubeFlow team wrote in a blog post. “Combining the simple orchestration of machine learning pipelines with RAPIDS, a collection of CUDA-accelerated libraries, data scientists can train and deploy machine learning pipelines significantly faster to solve business problems.”
