ISC 2020: Boosting Performance and Utilization with Multi-Instance GPU

DemoTeam, NVIDIA
ISC 2020
Multi-Instance GPU (MIG) on the NVIDIA A100 Tensor Core GPU can guarantee performance for up to seven jobs running concurrently on the same GPU—and each GPU instance is fully isolated with its own compute, memory, and bandwidth. This unique capability of the A100 GPU offers the right-sized GPU for every job and maximizes data center utilization. This demo shows inference performance on a single slice of MIG and then scales linearly across the entire A100.