As MONAI celebrates its fifth anniversary, we’re witnessing the convergence of our vision for open medical AI with production-ready enterprise solutions.
This announcement brings two exciting developments: the release of MONAI Core v1.4, expanding open-source capabilities, and the general availability of VISTA-3D and MAISI as NVIDIA NIM microservices. This dual release reflects our commitment to both the research community and clinical deployment.
Five years of community innovation
The MONAI journey has been remarkable, with over 3.5M downloads and more than 1K published papers showcasing its impact on medical AI research.
What began as a collaboration between NVIDIA and King’s College London has grown into a vibrant ecosystem supported by leading institutions worldwide. Organizations like GSK, DKFZ, Bristol Myers Squibb, Alara, Quantiphi, Dataiku, and Newton’s Tree have contributed invaluable expertise and resources, helping shape MONAI into the standard platform for medical imaging AI.
MONAI Core v1.4: Advancing open medical AI
The latest release of MONAI Core features a range of new algorithmic capabilities as well as three foundation models that demonstrate the versatility and power of the MONAI framework.
Foundation models VISTA-3D, VISTA-2D, and MAISI
VISTA-3D is a specialized interactive foundation model that excels at annotating human anatomies from 3D CT images. Its two primary features are accurate out-of-the-box performance covering over 126 anatomy classes, and zero-shot ability to learn to segment novel structures with interactive annotation.
The model and its implementation details are available in the VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging paper, and you can access the model through the MONAI Model Zoo. Experience VISTA-3D’s interactive capabilities directly through the NVIDIA API Catalog, or deploy it as a portable, scalable NVIDIA NIM microservice.
VISTA-2D introduces capabilities in microscopy analysis for cell biology researchers. It uses a transformer network architecture with ~100M parameters that builds upon Meta’s Segment Anything Model (SAM) architecture. It handles multiple cell types and imaging modalities with high accuracy.
The model’s extensive validation across datasets like TissueNet, LIVECell, and Cellpose demonstrates its performance in real-world applications. For more information, see Advancing Cell Segmentation and Morphology Analysis with NVIDIA AI Foundation Model VISTA-2D and access the implementation through MONAI Model Zoo.
MAISI (Medical AI for Synthetic Imaging) represents our venture into 3D generative AI for healthcare. This latent diffusion model can create high-resolution 3D CT images up to 512 × 512 × 768 voxels, with voxel sizes ranging from 0.5mm to 5.0mm, and with user control of 10 body habitus classes complete with corresponding segmentation masks.
Key modules originally developed in the MONAI Generative repository have been integrated into MONAI Core codebase and are used in the development of MAISI. For more information, see MAISI: Medical AI for Synthetic Imaging, access the model through MONAI Model Zoo, or experience it directly through the NVIDIA API Catalog.
Enterprise-ready NIM microservices: VISTA-3D and MAISI
Building on the success of these open-source models, we’re excited to announce the general availability of VISTA-3D and MAISI as NVIDIA NIM microservices.
These production-ready solutions are delivered as part of NVIDIA AI Enterprise, providing containerized, GPU-accelerated inference capabilities that can be deployed across clouds, data centers, and workstations with a single command.
The NIM microservice implementations of VISTA-3D and MAISI offer several key advantages for enterprise deployment:
- Optimized inference engines using NVIDIA TensorRT for best performance
- Industry-standard APIs for seamless integration into existing medical imaging workflows
- Optimization of response latency and throughput based on the GPU system
- Flexible deployment options across various infrastructure environments
- Enterprise-grade security and scalability features
These medical AI models benefit from the NIM architecture. Organizations can maintain complete control of their applications and data while using NVIDIA performance optimization expertise through these containerized deployments.
For developers and healthcare organizations looking to integrate these capabilities, we provide comprehensive documentation, deployment guides, and support resources through the NVIDIA AI Enterprise platform.
Whether you’re implementing these models in a research environment or scaling them for clinical workflows, NIM microservices provide the infrastructure foundation needed for deployment.
M3: The future of medical visual language models
In parallel with our core development, we’re excited to share M3 (MONAI multi-modal model), a research initiative exploring the future of medical AI. This framework bridges the gap between visual understanding and natural language processing, demonstrating how foundation models can work together with specialized expert models.
Through VILA-M3, we’re showcasing how AI systems can leverage multiple types of expertise when analyzing medical images. For example, when querying about tumors in an MRI scan, the system can automatically trigger relevant segmentation models to enhance its analysis. VILA-M3 is available on Huggingface in multiple model parameter sizes to support various deployment needs.
For a complete overview of MONAI v1.4’s features, see the v1.4 release notes. You can also see these capabilities in action through our recordings of the MONAI Days 2024 presentations, which showcase VISTA-3D, VISTA-2D, VILA-M3, and MAISI.
MONAI Days 2024: Community in action
This year’s MONAI Days, held in partnership with the MICCAI Society, showcased the vibrant spirit of our community. The presentation topics were expanded to include the open-source Holoscan SDK.
On the first day, with 300 registrants, Holoscan SDK was presented as the open-source foundation to MONAI Deploy. Holoscan SDK enables MONAI models to be easily integrated into radiological workflows as well as into real-time surgical guidance and surgical robotics systems.
provider: youtube
url: https://youtu.be/8cTjMtGdwZE?list=PLtoSVSQ2XzyDOjOn6oDRfEMCD-m-Rm2BJ&t=1
src: https://www.youtube-nocookie.com/embed/8cTjMtGdwZE?list=PLtoSVSQ2XzyDOjOn6oDRfEMCD-m-Rm2BJ
src gen: https://www.youtube-nocookie.com/embed/8cTjMtGdwZE?start=1&list=PLtoSVSQ2XzyDOjOn6oDRfEMCD-m-Rm2BJ
On the second day, with over 500 registrants, attendees listened to researchers talking about the latest contributions to MONAI, metrics for quantifying validation performance, and the generative AI and M3 capabilities announced in the MONAI Core v1.4 release.
During this two-day event, many of the questions asked by attendees were answered by other attendees. The MONAI community has become self-sustaining, not only in terms of state-of-the-art methods being contributed to its codebase but also in terms of the support it offers to new users and developers.
All presentations are available on the MONAI YouTube channel. The genuine collaboration was evident as experienced community members stepped up to answer questions from newcomers, demonstrating the supportive environment we’ve built together.
Shaping the next 5 years of medical AI
As you look toward MONAI’s future, you’ll see a few key directions driving our innovation. Each builds on the current momentum while pushing into new territories critical for the advancement of medical AI.
Expanding clinical impact
We’re focusing on bridging the gap between research and clinical deployment, working closely with MedTech partners to integrate MONAI capabilities into everyday medical workflows. This includes expanding our enterprise-ready solutions and streamlining the path from development to deployment.
Multi-modal integration
The future of medical AI lies in combining multiple data streams, from medical imaging and electronic health records to real-time sensor data.
Building on initiatives like M3, we’re working to create comprehensive systems that can understand and integrate diverse medical data types.
Real-time processing evolution
Through integration with technologies like Holoscan SDK, we’re pushing the boundaries of real-time medical AI processing, enabling applications from surgical guidance to immediate diagnostic assistance.
Join us in building the future
Together, we’re creating more than just software—we’re building an ecosystem where medical AI innovation thrives, benefiting researchers, clinicians, and patients worldwide. For more information and to become part of the journey, see MONAI.
