AI Red Team

Sep 18, 2024

NVIDIA Presents AI Security Expertise at Leading Cybersecurity Conferences

Each August, tens of thousands of security professionals attend the cutting-edge security conferences Black Hat USA and DEF CON. This year, NVIDIA AI security...
9 MIN READ
An illustration showing a securit alert.
Jul 11, 2024

Defending AI Model Files from Unauthorized Access with Canaries

As AI models grow in capability and cost of creation, and hold more sensitive or proprietary data, securing them at rest is increasingly important....
6 MIN READ
Jun 27, 2024

Secure LLM Tokenizers to Maintain Application Integrity

This post is part of the NVIDIA AI Red Team’s continuing vulnerability and technique research. Use the concepts presented to responsibly assess and increase...
6 MIN READ
Nov 15, 2023

Best Practices for Securing LLM-Enabled Applications

Large language models (LLMs) provide a wide range of powerful enhancements to nearly any application that processes text. And yet they also introduce new risks,...
11 MIN READ
Picture of the ML security training classroom at Black Hat USA
Oct 19, 2023

NVIDIA AI Red Team: Machine Learning Security Training

At Black Hat USA 2023, NVIDIA hosted a two-day training session that provided security professionals with a realistic environment and methodology to explore the...
4 MIN READ
Oct 04, 2023

Analyzing the Security of Machine Learning Research Code

The NVIDIA AI Red Team is focused on scaling secure development practices across the data, science, and AI ecosystems. We participate in open-source security...
12 MIN READ
Aug 04, 2023

Mitigating Stored Prompt Injection Attacks Against LLM Applications

Prompt injection attacks are a hot topic in the new world of large language model (LLM) application security. These attacks are unique due to how ‌malicious...
10 MIN READ
Aug 03, 2023

Securing LLM Systems Against Prompt Injection

Prompt injection is a new attack technique specific to large language models (LLMs) that enables attackers to manipulate the output of the LLM. This attack is...
15 MIN READ
Two men working at a desktop computer in an office.
Jun 14, 2023

NVIDIA AI Red Team: An Introduction

Machine learning has the promise to improve our world, and in many ways it already has. However, research and lived experiences continue to show this technology...
12 MIN READ
Feb 13, 2023

Evaluating the Security of Jupyter Environments

How can you tell if your Jupyter instance is secure? The NVIDIA AI Red Team has developed a JupyterLab extension to automatically assess the security of Jupyter...
7 MIN READ
Letters, numbers, and padlocks on black background
Nov 30, 2022

Improving Machine Learning Security Skills at a DEF CON Competition

Machine learning (ML) security is a new discipline focused on the security of machine learning systems and the data they are built upon. It exists at the...
8 MIN READ