Researchers in the Robot Learning Lab at Cornell University developed a robot that can prepare a cup of latte without ever having seen the machine before – the robot does this by visually observing the machine and by reading online instruction manuals, similar to how humans learn.
The team used CUDA and TITAN X GPUs to train their deep learning models and then also uses the GPU during testing where the robot is trying to infer which human manipulation motion is best suited for the object it has never seen before.
“We use a deep learning neural network that can tell the robot which action in a database is the closest to the one it has to perform,” said researcher Jaeyong Sung. The researchers turned to the Amazon Mechanical Turk crowdsourcing service to collect a large library of actions — they invited hundreds of visitors to guide a robot through the motions to perform various tasks described in a set of printed instructions. To make crowdsourcing possible, the researchers created a web interface that lets the user guide an imaginary robot arm, almost like playing a video game.
You can help teach the robot by visiting the student’s website.
Read more >
AI-Generated Summary
- Cornell University researchers developed a robot that prepares a latte by visually observing an unfamiliar machine and reading online instruction manuals, learning in a manner similar to humans.
- The team trained deep learning models using CUDA and TITAN X GPUs, then used the GPU during testing to infer the best manipulation motion for previously unseen objects.
- A neural network identifies which action in a crowdsourced database most closely matches the required task, with hundreds of Amazon Mechanical Turk participants guiding a simulated robot arm through a web interface to build that library.
Next Steps
- Visit the Robobarista project website to help teach the robot new tasks.
- Read the Robobarista research paper for detailed methodology and results.
Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more
About the Authors
