NVIDIA TensorRT for RTX
TensorRT for RTX brings optimized AI inference and cutting-edge acceleration to developers using NVIDIA RTX GPUs. Offering peak performance for PC AI workloads such as CNN, Transformers, Speech & Diffusion models, and engineered to be lean at under 200MB, TensorRT for RTX delivers fast engine build times, typically within 15 to 30s. Engines built with TensorRT for RTX are portable across GPUs and OS – allowing build once, deploy anywhere workflows.
TensorRT for RTX supports NVIDIA GeForce and RTX GPUs from the Turing family all the way to Blackwell and beyond. SDKs can be available for both Windows and Linux development.
Please review TensorRT for RTX documentation for more information and visit our GitHub for samples and demos.
Available Versions
TensorRT for RTX 1.1 (Windows)
TensorRT for RTX 1.1 (Linux)
Notable changes in this TensorRT-RTX release:
- Added the IRuntime::getEngineValidity() API to programmatically and efficiently check whether a TensorRT-RTX engine file is valid on the current system or needs to be rebuilt due to incompatibilities in the software version, compute capability, and so on.
- Compilation time has been greatly improved, particularly for models with many memory-bound kernels. On average a 1.5x improvement is observed across a variety of model architectures.
Available Versions
TensorRT for RTX 1.0 (Windows)
TensorRT for RTX 1.0 (Linux)
This TensorRT-RTX release includes the following key features and enhancements when compared to NVIDIA TensorRT.
- Reduced binary size of under 200 MB for improved download speed and disk footprint when included in consumer applications.
- Splitting optimization into a hardware-agnostic "ahead-of-time" (AOT) phase and a hardware-specific "just-in-time" (JIT) phase in order to improve user experience. Completes end-2-end engine compilation in under 30s
- Improved adaptivity to real-system resources for applications where AI features run in the background to for eg: graphics
- Focused improvement on portability and deployment while still delivering industry-leading performance.
Ethical AI
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.