Clinical applications for AI are improving digital surgery, helping to reduce errors, provide consistency, and enable surgeon augmentations that were previously unimaginable.
In endoscopy, a minimally invasive procedure used to examine the interior of an organ or cavity of a body, AI and accelerated computing are enabling better detection rates and visibility.
Endoscopists can investigate symptoms, make a diagnosis, and treat patients by cauterizing a bleeding blood vessel, for example. There are numerous forms of endoscopy, many of them focused on gastroenterological diseases that affect the digestive tract.
Colonoscopy, one of the most common forms of gastrointestinal endoscopy, is essential for catching colorectal cancer, a disease that the American Cancer Society predicts will affect over 150,000 people in 2022.
With the assistance of AI, surgeries like endoscopy are becoming safer and more consistent while reducing surgeon workload. The tasks being augmented with machine learning algorithms include labeling, clearing surgical smoke, classifying airway diseases, identifying airway sizes, identifying lesions and diseased tissue, and auto-calculating the best physical routes for instruments.
To enable these clinical applications, technical algorithms are being developed for specific tasks:
- Organ segmentation for detection and automatic measurements
- Tool tracking
- Tissue type identification
- Optical flow
- Legion classification
Enhancing and processing video streams
In endoscopy, the task of enhancing and processing video streams is key to augmenting surgeon technical skills. This includes tasks of endoscopic image denoising, anomaly object detection, and anomaly measurements, as well as the streaming tasks of ingesting high-resolution and high-bandwidth data.
To implement AI and accomplish these tasks, developers must address numerous challenges in the medical device development process such as:
- Ingesting high resolution, high-bandwidth data streams
- Running AI inference with a low-latency budget
- Finding flexible sensor and data I/O options
- Building a distributed compute platform from edge to data center to cloud
- Adopting new deep learning algorithms
Today, most developers have to build individual solutions every time they have to solve a problem or workflow bottleneck. NVIDIA Clara Holoscan is a development platform with compute for AI workloads. With workloads such as enhanced visualization and automatic anomaly detection, you can easily customize solutions for various challenges.
Having an accelerated platform to iterate and integrate AI into endoscopy workflows gives you a low-risk, low-cost approach for adding augmentations to your existing endoscopy systems.
Whether deploying latency-sensitive, real-time tasks on the edge or analytic and summarization tasks to the cloud, NVIDIA Clara Holoscan offloads complexity, allowing you to quickly build custom AI solutions to improve endoscopy.
Endoscopy AI sample application on NVIDIA Clara Holoscan
Reference applications can provide an easy starting point if you’re looking to build custom applications for medical devices. The NVIDIA Clara Holoscan SDK includes a sample AI-enabled endoscopy application as a template for reusing components and app graphs in existing applications to build custom AI pipelines.
The endoscopy AI sample application has the end-to-end functionality of GXF, a modular and extensible framework to build high-performance applications. GXF provides support for devices that interface with AJA with an HDMI input. Its deep learning model can perform object detection and tool tracking in real time on an endoscopy video stream.
Several features are used to minimize the overall latency:
- GPUDirect RDMA video data transfer to eliminate the overhead of copying to or from system memory.
- NVIDIA Performance Primitive library for CUDA-accelerated 2D image transformations before AI inference.
- TensorRT runtime for optimized AI Inference and speed-up.
- CUDA and OpenGL interoperability, which provides efficient resource sharing on the GPU for visualization.
For more information about the endoscopy AI sample application, its hardware and software reference architecture on NVIDIA Clara Holoscan, as well as the path to production, see the Clara Holoscan Endoscopy whitepaper.
Featured image
An endoscopy image from a gallbladder surgery showing an AI-powered frame-by-frame tool identification and tracking. Image courtesy of Research Group Camma, IHU Strasbourg, and University of Strasbourg.
