A tale of two interactions: Inferring performance in hospitality encounters from cross-situation social sensing

Details

Serval ID
serval:BIB_15D62303119D
Type
Article: article from journal or magazin.
Collection
Publications
Institution
Title
A tale of two interactions: Inferring performance in hospitality encounters from cross-situation social sensing
Journal
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Author(s)
Muralidhar S., Schmid Mast M., Gatica-Perez D.
ISSN
2474-9567
Publication state
Published
Issued date
2018
Peer-reviewed
Oui
Volume
2
Number
3
Language
english
Abstract
People behave differently in different situations. With the advances in ubiquitous sensing technologies, it is now easier to capture human behavior across multiple situations automatically and unobtrusively. We investigate human behavior across two situations that are ubiquitous in hospitality (job interview and reception desk) with the objective of inferring performance on the job. Utilizing a dataset of 338 dyadic interactions, played by students from a hospitality management school, we first study the connections between automatically extracted nonverbal cues, linguistic content, and various perceived variables of soft skills and performance in these two situations. A correlation analysis reveals connection between perceived variables and nonverbal cues displayed during job interviews, and perceived performance on the job. We then propose a computational framework, with nonverbal cues and linguistic style from the two interactions as features, to infer the perceived performance and soft skills in the reception desk situation as a regression task. The best inference performance, with R2 = 0.40, is achieved using a combination of nonverbal cues extracted from the reception desk setting and the human-rated interview scores. We observe that some behavioral cues (greater speaking turn duration and head nods) are positively correlated to higher ratings for all perceived variables across both situations. The best performance using verbal content is achieved by fusion of LIWC and Doc2Vec features with R2 = 0.25 for perceived performance. Our work has implications for the creation of behavioral training systems with focus on specific behaviors for hospitality students.
Keywords
Applied computing, Human-centered computing, Social computing, First impressions, Hospitality, Nonverbal behavior, Multimodal interaction, Hirability, Job performance, Reception desk
Create date
11/10/2018 14:12
Last modification date
21/08/2019 5:16
Usage data