Jetson Project of the Month: OpenDog, a Gesture Controlled Robot

James Bruton of XRobots was awarded the ‘Jetson Project of the Month’ for OpenDog V2. This project uses the NVIDIA Jetson Nano Developer Kit to recognize hand gestures and control a robot dog without a controller. 

James, a robot inventor, thought it’d be nice if his OpenDog robot responded to hand gestures. To make this happen, he used transfer learning to retrain an existing SSD-Mobilenet object detection model using PyTorch. During the training process, he identified five hand gestures for the robot to move forward, backward, left, right and to jump. Using the camera capture tool, he captured these gestures and assigned them to the appropriate class. 

Capturing and labeling hand gesture data

He ensured that these images were captured at a specific distance from the camera to make sure the OpenDog doesn’t get distracted by hand gestures or similar patterns in the background.

James notes that the project can be improved by adding more training data which includes gestures in different indoor and outdoor backgrounds and from different users. Furthermore, he plans to convert OpenDog to a ROS robot similar to his Really Useful AI Robot. He created a series of videos to show his journey of building this project and the code is available on GitHub.

Discuss (0)