A researcher from Carnegie Mellon University developed S.A.R.A. (Socially Aware Robot Assistant) that not only comprehends what you say, but also understands facial expressions and head movements.
Using CUDA, GTX 1080 GPUs and cuDNN with TensorFlow to train the deep learning models, S.A.R.A. will reply differently if she detects a smile than someone frowning and offering a response that doesn’t comply with social norms.
“She looks at your behavior and her behavior,” says Justine Cassell, director of human-computer interaction at Carnegie Mellon University and director of the project, “and calculates on the basis of the intersection of that behavior. Which no one has ever done before.”
The S.A.R.A. project consists of three elements never before used, says Cassell: conversational strategy classifiers, a rapport estimator, and a social reasoner. “The conversational strategy classifiers are five separate recognizers that can classify any one of five conversational strategies with over 80 percent accuracy.”
Read more >
Virtual Agent Understands Your Social Cues
Nov 03, 2016
Discuss (0)
AI-Generated Summary
- Carnegie Mellon University researchers developed S.A.R.A., a Socially Aware Robot Assistant that comprehends speech while interpreting facial expressions and head movements.
- The system uses CUDA, GTX 1080 GPUs, and cuDNN with TensorFlow to train deep learning models that adjust responses based on detected social cues.
- S.A.R.A. incorporates three novel elements: conversational strategy classifiers achieving over 80 percent accuracy, a rapport estimator, and a social reasoner that calculates responses from the intersection of user and system behavior.
Next Steps
- Read the Popular Science feature on S.A.R.A. for additional details on the project.
Powered by NVIDIA Nemotron. AI-generated content may summarize information incompletely. Verify important information. Learn more
About the Authors
