Nonverbal Interpersonal Communication Exploration Toolbox

Gökce Ergün, Timo Lübbing, Senya Polikovsky, Carolin Schmitt

NICE Toolbox is an easy-to-use framework for exploring nonverbal human communication. It aims to enable the investigation of observable signs that reflect the mental state and behaviors of the individual. Additionally, these visual nonverbal cues reveal the interpersonal dynamics between people in face-to-face conversations.

NICE Toolbox incorporates a growing set of Computer Vision algorithms to track and identify important visual components of nonverbal communication. Existing state-of-the-art deep-learning and rule-based algorithms are combined into a single, easy-to-use software toolbox. Based on single- or multi-camera video data, the initial release encompasses whole-body pose estimation and gaze tracking for each individual as well as movement dynamics calculation (kinematics), gaze interaction monitoring (mutual-gaze), and the measurement of physical body distance between dyads. This first set of components and algorithms is going to be extended in future releases.

Click the image to start the video. 

In future releases, we plan to extend the NICE Toolbox to include detectors for facial expressions, head movements, eye closure, active speaking, emotional valence and arousal, and micro-action recognition.