Download              Webinar              Documentation              Forums             

The NVIDIA® Isaac Software Development Kit (SDK) is a developer toolbox for accelerating the development and deployment of AI-powered robots. The SDK includes the Isaac Robot Engine, packages with high-performance robotics algorithms, hardware reference applications and powerful simulation platform, Isaac Sim. The SDK accelerates robot development for manufacturers, researchers and startups by making it easier to add Artificial Intelligence (AI) for perception and navigation into modern day robots.

Isaac SDK capabilities:

  • Deep neural networks and GPU accelerated algorithms for robotics.
  • Develop in simulation and deploy easily in reality.
  • Integration of NVIDIA AI capabilities for robotic applications.
  • Easy bring up of sensors used for robotics applications.
  • Visualization of robot development in real and virtual worlds.
  • Interface with robotic platforms such as cobots (collaborative robots), AMRs (autonomous mobile robots), and other service/industrial robots.

ROBOT ENGINE

The Isaac Robot Engine is a software framework to easily build modular robotics applications. It enables high-performance data processing and deep learning for intelligent robots. Robotics applications developed on Isaac Robot Engine can seamlessly run on edge devices like the NVIDIA® Jetson AGX Xavier™ and NVIDIA® Jetson Nano™, as well as on a workstation with a discrete NVIDIA® GPU.

Entity-Component Architecture

  • The Robot Engine allows developers to break down a complex robotics use case into smaller building blocks.
  • Customize Isaac SDK features to fit your use case by configuring pre-packaged components.
  • Easily add new features by developing custom components.
  • Other capabilities include experimental message recording and replay of sensor data or message passing over network sockets.

Computational Graph

  • Applications consist of multiple nodes and use inter-node communication to exchange data between them.
  • Avoid host-device memory copies and increase the performance of your application by attaching CUDA buffer objects to messages.
  • Group nodes into subgraphs and effectively combine them into a robotics application.

Visualization

  • The Isaac Robot Engine also comes with a customizable visualization framework to create visualization for variable plots, drawings, camera image overlays, or 3D scenes.
  • Developers can use Isaac WebSight to inspect and debug their robotics application in a web browser.

Advanced Build System

  • Isaac SDK uses Bazel for tracking dependencies, building source code, testing and packaging of robotics applications.
  • Automatically pull external dependencies such as OpenCV, ROS or other robotics open source libraries.
  • Partners can create their own workspaces to add sensor support or new algorithms to Isaac SDK. An example is available on GitHub

Distributed Workspaces

WebSight

What's New for Robot Engine in 2019.2
  • Introducing distributed workspaces for Isaac SDK users and partners to create compatible, easily manageable workspaces and contribute to the growing Isaac community. Velodyne Lidar example available. Get the example from Isaac GitHub repository.

GEMS

GEMS are modular capabilities for sensing, planning, or actuation that can be easily plugged into a robotics application. For example, developers can add obstacle detection, stereo depth estimation, or human speech recognition to enrich their robot use cases. Similarly, developers can use the Isaac navigation stack, a collection of modules to build a robotics navigation stack, to their robot. GEMS capabilities currently supported by Isaac SDK:

Perception

  • Stereo Depth, Stereo Visual Odometry, Object/AprilTag Detection, Path Seg DNN, CUDA Orb, Superpixels

Navigation

  • Lidar based Global Localization, LQR Path Planning, Support for non-circular robot bases

IX - Human Machine Interaction

  • Audio Keyword Detection

Robot Platforms

  • Segway RMP210

Sensors

  • Stereo Camera, Structured Light Depth Camera, Velodyne VLP16 lidar, IMU

What's New for GEMS in 2019.2
  • New capabilities for perception tasks:
    • Path Segmentation DNN
    • CUDA ORB Features
    • CUDA Superpixels
  • New navigation capability with support for non-circular robot bases
  • New sensor support for Livox Lidar

Isaac SIM

Isaac Sim is a virtual robotics laboratory and a high-fidelity 3D world simulator. It accelerates research, design, and development in robotics by reducing cost and risk. Developers can quickly and easily train and test their robots in detailed, highly realistic scenarios.

Developers can use virtual robots with simulated sensors (RGB, stereo, depth, segmentation, LIDAR, IMU) in Isaac Sim to test their applications in a high-fidelity simulation environment. Once tested, applications can be deployed to NVIDIA® Jetson AGX Xavier™, Jetson™TX2, or Jetson Nano™ running on physical robots.

Robots in Isaac Sim are tightly coupled to the tools and frameworks in Isaac SDK, enabling easy transfer of algorithms and data between physical and virtual robots.

Game Engines Supported

  • Unreal Engine
  • Unity 3D

Domain Randomization

Using synthetic data and controlling the randomization parameters in a scene, developers can create huge customized data sets. In Isaac Sim, developers can seamlessly randomize lighting, object materials, object colors, object poses and camera properties.

Robot Builder

The most common robot model is URDF. Isaac Sim has an embedded URDF loader which allows you to load the URDF model of your robot into Isaac Sim and simulate its joints and movements. The URDF models for several mobile bases and manipulators have been tested in Isaac Sim.

What's New for Simulation in 2019.2
  • Introducing Isaac Sim support for Unity 3D Game Engine, popular for building scenarios without the need for in-house 3D expertise.

REFERENCE DESIGN & APPLICATIONS

Kaya the robot to get started with robotics

Kaya

People who would like to get started with Isaac SDK can build their own small robot platform using Kaya reference robot platform.

Kaya can be built using off-the-shelf components and 3D printed parts. The low-cost Kaya robot takes advantage of the new NVIDIA® Jetson Nano™. Samples applications built for the platform include April tag detection, obstacle avoidance, remote operation, object classification based on Yolo and much more. Some of the included apps are:

  • Follow-me
  • Mapping
  • Detect/Classify
Documentation

Carter the delivery robot

Carter

Carter is an Isaac SDK reference robot platform for autonomous indoor delivery and logistics based on NVIDIA Jetson platform. Developers can build their own Carter robot using sample applications. With NVIDIA Isaac SDK, developers can add capabilities for localization, autonomous navigation, map edition, path planning and state machines. Some of the included apps are:

  • Map Creation
  • Map Waypoint
  • Pose
  • Patrol
  • Random
Documentation

RESOURCES