Researchers in the Robot Learning Lab at Cornell University developed a robot that can prepare a cup of latte without ever having seen the machine before – the robot does this by visually observing the machine and by reading online instruction manuals, similar to how humans learn.
The team used CUDA and TITAN X GPUs to train their deep learning models and then also uses the GPU during testing where the robot is trying to infer which human manipulation motion is best suited for the object it has never seen before.
“We use a deep learning neural network that can tell the robot which action in a database is the closest to the one it has to perform,” said researcher Jaeyong Sung. The researchers turned to the Amazon Mechanical Turk crowdsourcing service to collect a large library of actions — they invited hundreds of visitors to guide a robot through the motions to perform various tasks described in a set of printed instructions. To make crowdsourcing possible, the researchers created a web interface that lets the user guide an imaginary robot arm, almost like playing a video game.
You can help teach the robot by visiting the student’s website.
Read more >
Meet the GPU-Accelerated Latte-Making Robot
Oct 12, 2016
Discuss (0)

Related resources
- GTC session: Accelerating Massive Route Planning for Food Delivery with GPU (Spring 2023)
- GTC session: Accelerating Robotics from the Edge to Cloud with AI and Simulation (Spring 2023)
- GTC session: Optimizing Applications for Hopper Architecture (Spring 2023)
- Webinar: NVIDIA Inception: Maximize Your Benefits
- Webinar: AI on the Edge for Robotics and Automation in Agriculture
- Webinar: Accelerating Data Science Workflows with RAPIDS