GTC 2020: Training Financial Language Models on Bloomberg's Kubernetes Data Science Platform
After clicking “Watch Now” you will be prompted to login or join.
Click “Watch Now” to login or join the NVIDIA Developer Program.
Training Financial Language Models on Bloomberg's Kubernetes Data Science Platform
Ian Hummel, Bloomberg | CJ Zheng, Bloomberg
We'll explain how we leverage the Bloomberg Data Science Platform to accelerate training large neural networks on our enormous in-house corpora. We'll discuss how we integrate open-source solutions, such as Kubernetes and KubeFlow, and review our results from pretraining BERT entirely from scratch. The Bloomberg Terminal provides data, analytics, news, and communication for professionals in business, finance, government, and philanthropy. Natural language processing (NLP) plays a growing role in our toolkit due to the increasing importance of textual data in modern finance. Transformer models such as BERT, XLNet and OpenAI's GPT series achieve superior performance on a wide variety of NLP tasks by replacing recurrence of neurons with self-attention mechanisms. At Bloomberg, we train customized transformer LMs on finance-related corpora to improve performance of production tasks like named entity recognition, document classification, question answering, and more.