Technical Walkthrough 0

Accelerating Inference with Sparsity Using the NVIDIA Ampere Architecture and NVIDIA TensorRT

○ TensorRT is an SDK for high-performance deep learning inference, and TensorRT 8.0 introduces support for sparsity that uses sparse tensor cores on NVIDIA Ampere GPUs. It can accelerate networks by reducing the computation of zeros present in GEMM operations in neural networks. You get a performance gain compared to dense networks by just following the steps in this post. 8 MIN READ
Technical Walkthrough 0

Real-Time Natural Language Processing with BERT Using NVIDIA TensorRT (Updated)

Today, NVIDIA is releasing TensorRT 8.0, which introduces many transformer optimizations. With this post update, we present the latest TensorRT optimized BERT sample and its inference latency benchmark on A30 GPUs. Using the optimized sample, you can execute different batch sizes for BERT-base or BERT-large within the 10 ms latency budget for conversational AI applications. 18 MIN READ
Technical Walkthrough 0

Speeding Up Deep Learning Inference Using NVIDIA TensorRT (Updated)

This post was updated July 20, 2021 to reflect NVIDIA TensorRT 8.0 updates. NVIDIA TensorRT is an SDK for deep learning inference. TensorRT provides APIs and… 22 MIN READ
News 0

NVIDIA Releases Updates to CUDA-X AI Software

NVIDIA CUDA-X AI is a deep learning software stack for researchers and software developers to build high performance GPU-accelerated applications for conversational AI, recommendation systems and computer vision. 6 MIN READ
News 0

Jetson Project of the Month: DeepWay, AI-based navigation aid for the visually impaired

Satinder Singh won the Jetson Project of the Month for DeepWay, an AI-based navigation assistance system for the visually impaired. The project… 2 MIN READ
News 0

NVIDIA Releases Riva 1.0 Beta for Building Real-Time Conversational AI Services

Jarvis is a flexible application framework for multimodal conversational AI services that delivers real-time performance on NVIDIA GPUs. 4 MIN READ