3D Recognizer

Quickly train and test your own segmentation models on live 3D point cloud data. With the deep learning model included in this project, you'll be able to detect fingertips from incoming video from an Intel RealSense L515 LiDAR camera, or from pre-recorded point cloud stream. The model (a PyTorch implementation of RandLA-Net) can run on Jetson using the Docker container included. The project UI displays a top section with three screens with 3D output which can be rotated / manipulated. The first screen is a live camera output from the point cloud footage. The middle screen allows capturing and annotating snapshot samples taken from the first screen. Once the annotating and training is complete, the third screen can be toggled to display the model's predictions every 250ms using live data from the first screen. The bottom part of the UI helps with the training done with the middle screen, and it can dynamically adjust the confidence interval used for the live predictions.