Using NVIDIA Nsight Compute in Containers

Containers are now ubiquitous, and for good reason; the portability and productivity enhancements they provide have made them a standard component in HPC and many other computing fields. The NVIDIA Nsight family of developer tools for analyzing performance of CUDA applications are supported in container environments. For more information about the environmental landscape and Nsight Systems, see Using NVIDIA Nsight Systems in Containers and the Cloud. This post contains similar information for using Nsight Compute in container environments. Nsight Compute is specifically designed to provide detailed performance analysis of CUDA kernels running on the GPU.

Setting up a system for profiling

Profiling with Nsight Compute requires access to the performance counters on the GPU. Whether or not you are running in a container, you may run into a permissions issue if they are not available. For more information about this common issue, see Permission issue with Performance Counters. These counters need to be accessible on the machine where the application is running and being profiled. This should be configured on the machine outside of the container.

Installing Nsight Compute inside a container

If your container does not have Nsight Compute installed already, you can install it just like you would outside of a container. You must have the NVIDIA repositories set up, which should be the default in any NGC container. For more information, see the following sections for sample commands to add the repositories manually to your existing containers. When you have the repos set up, use a command like these to install Nsight Compute:

# apt-get update -y
# apt-get install -y nsight-compute-2020.1.0

If you’re unsure of the versions available, use a command like the following:

# apt-cache search nsight-compute

Adding Nsight Compute to an existing Docker container

Adding Nsight Compute to your existing Docker container image is straightforward. The following code examples provide example lines to add to the existing Dockerfile. These examples assume Nsight Compute version 2020.1.0, which was the most recent at the time of publication. The available versions can be found using a command like apt-cache search nsight-compute, provided that the NVIDIA repositories are set up. Use the appropriate code example depending on whether your container image is RHEL-based or Debian-based.

Debian-based Docker

RUN apt-get update -y && \
     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
         apt-transport-https \
         ca-certificates \
         gnupg \
         wget && \
     rm -rf /var/lib/apt/lists/*
RUN  echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list && \
     wget -qO - https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub | apt-key add - && \
         apt-get update -y && \
     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
         nsight-compute-2020.1.0 && \
     rm -rf /var/lib/apt/lists/*

RHEL-based Docker

RUN rpm --import https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub && \
     yum install -y yum-utils && \
     yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64 && \
     yum list available | grep nsight && \
     yum install -y \
         nsight-compute-2020.1.0 && \
     rm -rf /var/cache/yum/*

Adding Nsight Compute to an existing Singularity container

Adding Nsight Compute to your existing Singularity container image is almost identical to Docker. The following code examples provide example lines to add to the existing Singularity file. These code examples assume Nsight Compute version 2020.1.0, which was the most recent at the time of publication. The available versions can be found using a command like apt-cache search nsight-compute. Use the appropriate code example depending on whether your container image is RHEL-based or Debian-based.

Debian-based Singularity

%post 
    apt-get update -y && \
     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
         apt-transport-https \
         ca-certificates \
         gnupg \
         wget && \
     rm -rf /var/lib/apt/lists/*
     echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list && \
     wget -qO - https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub | apt-key add - && \
         apt-get update -y && \
     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
         nsight-compute-2020.1.0 && \
     rm -rf /var/lib/apt/lists/*

RHEL-based Singularity

%post    
    rpm --import https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/7fa2af80.pub && \
     yum install -y yum-utils && \
     yum-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64 && \
     yum list available | grep nsight && \
     yum install -y \
         nsight-compute-2020.1.0 && \
     rm -rf /var/cache/yum/*

Mapping an Nsight Compute host installation into a container

So far, I’ve described how to add Nsight Compute to a Docker or Singularity container image. Sometimes, it may be easier to use the existing Nsight Compute installation from the host inside the container.

An Nsight Compute installation on the host can be mounted into a container at runtime but this may not work properly if the required Nsight Compute dependencies available on the host are not available inside the container.

For example, if the host Nsight Compute is installed on the host at /opt/nvidia/nsight-compute/2020.1.0/, you can add the following command-line options to temporarily make Nsight Compute available inside a container.

$ sudo docker run --gpus all  -it -v /opt/nvidia/nsight-compute/2020.1.0/:/opt/nvidia/nsight-compute/2020.1.0:ro …
$ singularity run --nv -B /opt/nvidia/nsight-compute/2020.1.0/:/opt/nvidia/nsight-compute/2020.1.0/ ...

Using the GUI from a container (Singularity only)

The previous sections described how to use the Nsight Compute command-line interface with containers. In those cases, the generated report file could be opened and visualized on another machine with the GUI. However, it is also possible to use the Nsight Compute GUI inside the container.

The container image must have several X11 and related packages installed to satisfy the Nsight Compute GUI requirements. The following is a reference Singularity definition file. There may be additional or duplicate requirements listed here depending on your container. It is provided to be instructive but can be modified as needed.

Bootstrap: docker
From: nvidia/cudagl:10.1-base-ubuntu18.04

%post
    apt-get update -y
    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        apt-transport-https \
        ca-certificates \
        dbus \
        fontconfig \
        gnupg \
        libasound2 \
        libfreetype6 \
        libglib2.0-0 \
        libnss3 \
        libsqlite3-0 \
        libxkbcommon-x11-0 \
        libx11-xcb1 \
        libxcb-glx0 \
        libxcb-xkb1 \
        libxcomposite1 \
        libxcursor1 \
        libxdamage1 \
        libxi6 \
        libxml2 \
        libxrandr2 \
        libxrender1 \
        libxtst6 \
        openssh-client \
        wget \
        xcb \
        xkb-data

      echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list && \
     wget -qO - https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub | apt-key add - && \
         apt-get update -y && \
     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
         nsight-compute-2020.1.0 && \
     rm -rf /var/lib/apt/lists/*

%environment
    export LC_ALL=C
    export PATH=/opt/nvidia/nsight-compute/2020.1.0/:$PATH
%runscript
    exec ncu-ui "$@"

 To create the Nsight Compute GUI container image, first create a file named Singularity.def with the earlier content and then build the image.

$ sudo singularity build ncu-gui.sif Singularity.def

Running the container automatically starts the Nsight Compute GUI:

$ singularity run --nv ncu-gui.sif

Call to action

If you’re writing CUDA kernels and you’re concerned about performance, Nsight Compute should be a tool in your toolbox. With support for containers, it’s easy to try and easy to integrate into your existing development environment.

To learn more, see the following resources:

Have a question? Post it to the NVIDIA forums using NVIDIA Nsight Compute or drop a message.