Technical Walkthrough

Jumpstarting AI with a COVID-19 CT Inference Pipeline and the NVIDIA Clara Deploy QuickStart Virtual Machine

Discuss (0)

Getting AI up and running in hospitals has never been more important. Until recently, connecting an inference pipeline to perform analysis has had its challenges and limitations. There is a considerable amount of complexity in setting up and maintaining the hardware and software, deployment, configuration, and all workflow steps in an AI inference research pipeline. NVIDIA Clara Deploy, an application framework for running imaging inference pipelines at scale, can be used to streamline this process.

In this post, we demonstrate the NVIDIA Clara Deploy QuickStart Virtual Machine (VM) using a COVID-19 inference pipeline. With the NVIDIA Clara VM and the following scripts, getting started is seamless. NVIDIA has put together a simple three-step process to get your hospital environment ready to deploy inference at scale:

  1. Deploy the base VM: Use a clean Ubuntu Linux 18.04 instance.
  2. Run the installation scripts: Download and execute the Clara Deploy Ansible scripts, currently in early access.
  3. Trigger the pipeline: Open the sample CT study with the embedded web client, run the job using the preinstalled Clara Deploy pipeline, and visualize the inference results.

A successful deployment of any deep learning-based application requires meaningful integration with healthcare information systems. The inference platform that is used to execute deep learning-based workflows need to support industry standards such as DICOM for ingestion of input data and publishing of inference results.

Clara VM is preconfigured to accept data from outside and can be integrated with an external PACS.

The Clara VM integrated with an external PACS.
Figure 1. Network topology diagram of the Clara VM.

Inside the Clara VM

The Clara VM relies on Clara Deploy, a framework for creating AI workflows, using pipelines and operators.

A diagram showing the 7 steps in a typical end-to-end Clara Deploy pipeline. This includes a PACS sending data over an interoperable transport mechanism, following by a receive adapter, pre-transforms, inference, post-transforms, and delivery via the Send Adapter.
Figure 2. A typical Clara Deploy workflow.

A Clara Deploy instantiation follows the same pattern:

  1. PACS: This system holds DICOM images acquired from modalities, where you configure a send destination to Clara.
  2. Interoperability transport: Within the PACS, DICOM images are sent routing rules or through manual pushes.
  3. Receive adapter: Clara Deploy DICOM Adapter receives the DICOM images.
  4. Pre-transforms: Clara Deploy converts DICOM imaging data into a format that the inference engine, TRITON, can process.
  5. Inference: NVIDIA Triton, part of Clara Deploy, performs inference on the imaging study to derive observations on the data.
  6. Post-transforms: The observations are transformed into a format, like a DICOM Segmentation, Structured Report (DICOM-SR), or HL7 FHIR message.
  7. Send adapter: Clara Deploy transmits these observation messages to the destination system, such as PACS.

All of this processing happens locally and no processing is performed externally. After it is set up, no connection to the internet is required. It is important that you apply the appropriate security measures that you would apply to any system that has potentially sensitive data.

In addition to Clara Deploy, there are other components included in Clara VM to accelerate the deployment experience: an image repository using Orthanc PACS, a DICOM web viewer with OHIF, and a sample study to demonstrate how the pipeline works.

Deploy the Clara VM

The following instructions walk you through the steps necessary to provision a VM, configure the Clara Deploy installation, and use the included Orthanc repository and and OHIF web viewer to visualize results of the Clara COVID Segmentation pipeline. Clara VM is currently in early access, which requires approval to access the scripts from NGC. Sign up for early access.

To get started, you only need two components from a hardware perspective:

  • Infrastructure to deploy a virtual machine on (for example, VMware, or a cloud instance)
  • Access to a NVIDIA GPU, such as NVIDIA T4, V100, or A100

The installation scripts assume that you have access to a system with a passthrough T4, 16 vCPU, 32 GB RAM, and a 500-GB disk. If you have something different, some adjustments may need to be tweaked.

Step 1: Deploy the base VM

To streamline this process, you can download a VM ISO with the platform preconfigured, allowing you to begin interacting with the platform immediately.

Deploy a basic Ubuntu OS on the VM instance.

Step 2: Run the installation script

Configuration of the VM is done using Ansible, a tool for repeatably configuring bare metal or virtualized machines. Ansible can be used either locally or from a separate provisioning system. In this case, use a local installation, which requires a few simple prerequisites.

First, you update, install dependencies, add the required repositories, and then install Ansible:

sudo apt update
sudo apt install -y software-properties-common unzip
sudo apt-add-repository --yes --update ppa:ansible/ansible
sudo apt install -y ansible

To pull the Ansible scripts for installing and configuring the NVIDIA driver and Clara Platform, you must install the NGC CLI and configure it to use your NGC API key. For more information about installation instructions, including generating the NGC API key, see Introduction to the NGC Catalog and Registry CLIs. For this VM, install as follows:

mkdir ~/bin && cd ~/bin
wget -O ngccli_reg_linux.zip \
https://ngc.nvidia.com/downloads/ngccli_reg_linux.zip
unzip -o ngccli_reg_linux.zip
chmod u+x ngc
md5sum -c ngc.md5
echo "export PATH=\"\$PATH:$(pwd)\"" >> ~/.bash_profile
source ~/.bash_profile
ngc config set

The NGC config set command prompts for your NGC API key. Defaults may be accepted for the other configuration options. You can then use the NGC CLI to download the Clara Ansible scripts:

ngc registry resource download-version \
"ea-nvidia-clara/clara/clara_ansible:0.7.3-2011.5"
cd clara_ansible_v0.7.3-2011.5
unzip ansible.zip
cd playbooks

The playbooks directory contains the Ansible scripts to install and configure the NVIDIA driver and Clara Platform. This installation happens in three steps:

  • Install and configure the NVIDIA driver.
  • Install the Clara Platform.
  • Install and configure the COVID-19 pipeline.

If you already have the NVIDIA driver and Clara Deploy up and running, you can skip the first two steps and start with the pipeline installation.

To configure this for the local system, two configuration changes to the clara_hosts file are required:

  • Add the following entry for localhost under the [clara_hosts] section (one line):
    localhost ansible_connection=local ansible_python_interpreter="{{ansible_playbook_python}}"
  • Add your NGC API key in the last line of the file

First, install the NVIDIA driver:

ansible-playbook -K install-nvidia-driver.yml -i clara_hosts
sudo reboot  # manual reboot is required because you’re running locally

Then install and configure the Clara Platform:

ansible-playbook -K install-clara.yml -i clara_hosts

The final step is configuring the COVID-19 pipeline and Orthanc/OHIF endpoints.

ansible-playbook -K install-clara-pipeline.yml -i clara_hosts

These Ansible scripts download and install everything needed for Clara Deploy, as well as stage and start the COVID-19 “classification+lesion segmentation” pipeline, listen on a DICOM endpoint, and install Orthanc and OHIF for visualization of results. When the Ansible script is finished, it is ready to accept DICOM to automatically process and send the results into the local Orthanc for visualization in OHIF.

Step 3: Trigger the pipeline

An example COVID study has been registered with Orthanc. To view this study, navigate to the Orthanc interface at http://<IP of the VM>/pacs-admin/app/explorer.html.  This prompts you to log in. The Orthanc instance has been configured with username/password orthanc/orthanc. After logging in, you see the main Orthanc interface. To see all available studies, choose All Studies.

On the All Studies screen, you see the example Chest CT COVID-19 study that has been preloaded. Open to see the study details.

The Orthanc Patient Study interface showing the Axial Chest CT COVID-19 study.
Figure 3. The Orthanc overview of the preloaded Chest CT COVID-19 study.

In the left menu, choose Send to DICOM Modality, covid19. This triggers the Clara Deploy pipeline.

Review the results

After the pipeline has completed, refresh this view and you see that the study has been updated with the results of the pipeline, including the following:

  • Lung Segmentation
  • Lung Lesion Segmentation
  • Two DICOM reports
The Orthanc Patient Study interface showing the Axial Chest CT COVID-19 study after running the COVID-19 segmentation pipeline. In this view, you see the additional DICOM series for COVID-19 Segmentation and Classification reports.
Figure 4. The Chest CT COVID 19 study showing the segmentation and classification generated by the Clara COVID 19 pipeline.

This study can now be visualized in OHIF by navigating to http://<IP address of the VM>. In the main OHIF interface, choose Study List where you see the Chest CT COVID-19 study. Open this study to navigate to the visualization interface. You can view the lung and lesion segmentations along with DICOM reports that detail the results of COVID classification. Figure 6 shows an OHIF layout with panels showing the segmentation as well as the DICOM report with the COVID classification.

The integrated OHIV viewer showing the visualization of the Lung and Lesion segmentation and the classification report.
Figure 5. The OHIF visualization interface showing the lung lesion segmentation and classification report.

Considerations and next steps

When you are sending studies from your environment, only non-multiframe CT studies are accepted. These are denoted with the SOP Class 1.2.840.10008.5.1.4.1.1.2. Secondary captures may not be properly processed.

In the Clara VM for COVID, the Clara pipeline has been preconfigured with DICOM AE titles and source and destination endpoints to interface with Orthanc. To enable this kind of integration, under the hood, the Clara DICOM Adapter provides a set of tools to build this interface between an external PACS, like Orthanc, and the Clara Platform.

To send studies for processing to the Clara VM, perform a CSTORE to the IP address of the Clara VM, on port 4242, with an AE title of “CLARA”. This can be configured inside of a PACS by a PACS administrator by logging into the application and adding a send destination to the Clara VM. After the NVIDIA Clara DICOM Adapter receives the DICOM study, processing can begin.

Extending Clara VM with the COVID-19 Classification Pipeline

For this example, use the Clara Deploy AI COVID-19 Classification Pipeline available on NGC. Following the instructions there, pull and configure the pipeline:

mkdir -p ~/.clara/pipelines  && cd ~/.clara/pipelines
clara pull pipeline clara_ai_covid19_pipeline
cd clara_ai_covid19_pipeline
unzip app_covid-19-input_v1.zip -d input
sudo unzip app_covid-19-model_v1.zip -d /clara/common/models/
clara create pipeline -p COVID-19-pipeline.yaml

This results in a pipeline with a unique pipeline ID. You can create the DICOM source and destinations for Orthanc and then use this pipeline ID to create a DICOM AE title that is used to trigger the pipeline from Orthanc:

clara dicom create source -a ORTHANC -i 10.110.16.186
clara dicom create destination -a ORTHANC -i 10.110.16.186 \
-n MYPACS -p 4242
pipeid=$(clara list pipelines | grep COVID | awk '{print $2}')
clara dicom create aetitle -a covid19 \
pipeline-COVID-19-pipeline=${pipeid}

The COVID-19 pipeline includes a sample dataset that you can push to Orthanc. This is the dataset used in the preceding Orthanc/OHIF example. To push this dataset to Orthanc, use the DICOM toolkit as follows

storescu -v +sd +r -xb -v -aet DCM4CHEE \
-aec ORTHANC <IP of the VM> 4242 \
~/.clara/pipelines/clara_ai_covid19_pipeline/input/dcm/

This sample study is now accessible from the Orthanc UI, which can be used to trigger the pipeline using the covid19 AE title (modality), as in the preceding example.

This can be extended further to integrate into the medical imaging ecosystem. Follow the earlier outline to create a new pipeline using one of the example pipelines bundled with Clara Deploy. Following these steps, you can create a unique AE title, push an example dataset to Orthanc, trigger the pipeline, and visualize the results in OHIF.

You can also look more closely at the Clara Platform and pipelines using the Clara console. Install using the Clara CLI:

clara pull -y console
clara console start

After the Clara console is started, it is accessible at http://<IP of the VM>:32002. Use the console to view details of the pipelines and jobs that were triggered through the Orthanc interface. Figure 7 shows the Clara console Job view with the DAG for the COVID-19 pipeline.

Figure 6. The Clara Deploy console showing the DAG for the COVID-19 pipeline.

Figure 7. The Clara Deploy console showing the DAG for the COVID-19 pipeline.

Summary

In this post, we walked you through the steps to go from a base Ubuntu VM to a fully configured Clara Deploy Platform with the COVID-19 segmentation pipeline integrated with a PACS repository and results viewer. This serves as a blueprint for deploying applications within hospital infrastructure to enable Clara Deploy inference at scale.

For more information, request early access to Clara VM or see the Scalable and Modular AI Deployment Powered by NVIDIA Clara Deploy platform whitepaper.