Predicting the Speaker in Three-Party Conversations from Eye Contact Patterns
Imagine a future where robots can speak face-to-face with humans in a way that is entirely鈥ell, human. Accomplishing this requires a detailed understanding of behavior we take for granted.
鈥淧eople are very skilled at identifying the speaker in a conversation just from observing
their eye contact behavior,鈥� said Zhigang Deng, professor of computer science at the
探花视频. 鈥淨uantifying these patterns, however, is a challenge.鈥�
Computer-Human Interaction for Efficient Design of Technology
Deng鈥檚 research group works in a field known as computer-human interaction, which studies how we interact with computers. This field ranges from designing apps to voice recognition software. Understanding this interaction can lead to design of technology that feels intuitive for users.
On the forefront of this research is the challenge of designing robots that can interact with humans in a way that feels entirely natural. Accomplishing this, however, requires quantifying behavior, such as eye contact patterns, that we take for granted.
鈥淲e want robots to have the capability of identifying the speaker in a conversation,鈥� said Yu Ding, a postdoctoral researcher in Deng鈥檚 group, and the first author on a paper describing the research. 鈥淎 robot, which will be programmed by computer scientists, needs to know who is the speaker and the listener.鈥�
Predicting the Speaker in a Three-Party Conversation
Deng, along with his research group, developed a strategy to predict the speaker in a three-party conversation.
鈥淲e are the first group to use eye contact patterns and head orientation to identify the speaker in a three-party conversation,鈥� said Deng, who is part of the College of Natural Sciences and Mathematics.
The results will be presented at the ACM CHI Conference on Human Factors in Computing Systems, which is the top conference in their field. A copy of the manuscript describing these results is available on the website. Also included as co-authors on the paper are Yuting Zhang, a UH Ph.D. student in computer science, and Meihua Xhao, a visiting scholar.
鈥淓ye contact is constantly changing during a conversation. It鈥檚 very dynamic data,鈥� Ding said. 鈥淭he challenge is to extract meaningful behavior patterns.鈥�
Understanding Dynamic Data With Time-Series Analysis
To create this method, Deng鈥檚 group obtained a dataset of a lengthy conversation between three people, recorded using a high-quality optical motion capture system. These recordings, as well as the data analysis and modeling methodology developed by Deng鈥檚 group, made these predictions possible.
鈥淥ur methodology came down to time-series analysis,鈥� Ding said. 鈥淲hen we speak, our gaze changes all the time. That means we can鈥檛 detect the speaker by just looking at a single frame. Instead, we have to consider a sequence of frames in which the gaze of all three people is changing.鈥�
Although Deng is quantifying eye contact patterns during conversations for computer-human interactions, this knowledge is applicable to other disciplines.
For example, autism spectrum disorders are characterized by differences in eye contact behaviors. Although current diagnostic techniques are lengthy and subjective, a quantitative understanding of eye contact behaviors could offer doctors an additional tool for accurate diagnosis.
鈥淭hese research findings can be used in multiple fields,鈥� Deng said.
This research was supported by the National Science Foundation and the National Natural Science Foundation of China.
- Rachel Fairbank, College of Natural Sciences and Mathematics