Meetings' engagement is an observable behavior, often expressed by body cues. While we, as humans, are attuned to these cues to make sense of an in-person meeting., the picture is slightly different in a virtual meeting. Not only visual communication cues, but also non-verbal ones such as body postures and gestures might go unnoticed. Consider, for example, a virtual all-hands meeting, where one would expect a one-to-many form of communication. One could imagine that not everyone would feel comfortable sharing their video streams in such a setting and, as such, the speaker/presenter might miss the opportunity to `read the room' (i.e., interpret non-verbal cues expressed by meeting participants). Therefore, capturing body cues would not only bring them back to the virtual space, but also give meeting participants the opportunity to `read the virtual room'. While current meeting tools may well employ techniques for analyzing audio-visual or textual information in virtual meetings, the full spectrum of human senses is far from being captured. As the mobile-sensing technologies are growing in sophistication and ubiquitousness, we are now faced more than ever with a unique opportunity to measure human behavior that was previously impossible to measure.

Nokia Bell Labs researchers developed KAIROS, a system for multi-modal monitoring of virtual meetings. The system collects data from mobile, smartwatches, and earable devices which, in turn, are translated into six body cues metrics. These metrics capture: vibrancy (indicates a conversational meeting), multi-tasking (phone movements), heart rate, number of head movements, changes in postures, and hand and wrists movements. Using these six metrics, the researchers tested whether these metrics are predictive of meeting success. They developed a model that predicts meetings success with an AUC of 79%, and found that the most predictive metric was head movement (twice as important as hand movement). They have also shown that head movements were even more predictive than a meeting’s emotional content derived from the meeting’s transcript. Finally, they showed that these body cues were indicative of the presence (or absence) of three types of key phrases in a meeting that happened to greatly impact its success: the meeting's initiation, collective discussions, and turning points.

The scientists foresee that, in future, tools such as KAIROS could transform the ways we experience virtual meetings. Imposing physical distancing amid the COVID-19 pandemic, not only transformed the ways we communicate virtually but also affected our experience. To this end, KAIROS promises to bring back to the virtual world signals (i.e., body cues) that we are attuned to in physical, in-person interactions.

Publications

• KAIROS: Talking Heads and Moving Bodies for Successful Meetings. ACM HotMobile 2021 PDF