Can a bee teach an autonomous drone how to fly through gaps? Researchers from the University of Maryland’s Perception and Robotics Group recently developed a deep learning-based system that allows a drone to fly through a small and completely unknown gap automatically. They call it GapFlyt.
“We propose this framework of bio-inspired perceptual design for quadrotors. We use this philosophy to design a minimalist sensorimotor framework for a quadrotor to fly though unknown gaps without an explicit 3D reconstruction of the scene using only a monocular camera and onboard sensing,” the researchers stated in their paper.
Using NVIDIA TITAN Xp GPUs with MATLAB, the team compiled several pre-trained convolutional neural networks into one to develop their own. Their final version resulted in an optical flow network that runs on Python with TensorFlow. An NVIDIA Jetson TX2 GPU is mounted aboard the drone to run all the perception and control algorithms.
What makes this work unique is that the drone has no information about the location or size of the gaps in advance. It simply analyzes the visual information it sees and attempts to maneuver its way through slowly.
“We achieved a remarkable success rate of 85 % over 150 trials for different arbitrary shaped windows under a wide range of conditions which includes a window with a minimum tolerance of just 5cm,” the team said.
The drone reached a maximum speed of 2.5 meters per second while passing through the gap. However, with additional training and higher frame cameras, the team believes their method can achieve better results in the future.
The work was recently published on ArXiv, and the code has been published on GitHub.