DLRM ETL with GPU-Accelerated Spark 3 Early Access
Apache Spark is a leading framework for distributed scale-out data processing. With Spark, organizations are able to process large amounts of data, in a short amount of time, using a farm of servers—either to curate and transform data or to analyze data and generate business insights.
Given the “embarrassingly parallel” nature of dataset preprocessing for DLRM it’s only natural that the architecture of a GPU and the power of Spark’s distributed computing framework should be leveraged to achieve performance and cost savings similar to how GPUs have achieved in DL training workloads. With the RAPIDS Accelerator for Spark 3, GPU acceleration is transparent to the developer and requires no code changes in order to obtain these benefits.
This early access example solution provides a GPU-optimized DLRM in the form of a ready-to-go Docker image for training and inference. Included in the image are data downloading and preprocessing tools and Jupyter demo notebooks to get you up and running quickly. Trained models can then be prepared for production inference in one simple step with our exporter tool. We are excited to see what you can do with this model on your own data.