Revisiting the Deep Learning-Based Eavesdropping Attacks via Facial Dynamics from VR Motion Sensors

Soohyeon Choi, Manar Mohaisen, Daehun Nyang, David Mohaisen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Virtual Reality (VR) Head Mounted Display’s (HMD) are equipped with a range of sensors, which have been recently exploited to infer users’ sensitive and private information through a deep learning-based eavesdropping attack that leverage facial dynamics. Mindful that the eavesdropping attack employs facial dynamics, which vary across race and gender, we evaluate the robustness of such attack under various users characteristics. We base our evaluation on the existing anthropological research that shows statistically significant differences for face width, length, and lip length among ethnic/racial groups, suggesting that a “challenger” with similar features (ethnicity/race and gender) to a victim might be able to more easily deceive the eavesdropper than when they have different features. By replicating the classification model in [17] and examining its accuracy with six different scenarios that vary the victim and attacker based on their ethnicity/race and gender, we show that our adversary is able to impersonate a user with the same ethnicity/race and gender more accurately, with an average accuracy difference between the original and adversarial setting being the lowest among all scenarios. Similarly, an adversary with different ethnicity/race and gender than the victim had the highest average accuracy difference, emphasizing an inherent bias in the fundamentals of the approach through impersonation.

Original languageEnglish
Title of host publicationInformation and Communications Security - 25th International Conference, ICICS 2023, Proceedings
EditorsDing Wang, Zheli Liu, Moti Yung, Xiaofeng Chen
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages19
ISBN (Print)9789819973552
StatePublished - 2023
Event25th International Conference on Information and Communications Security, ICICS 2023 - Tianjin, China
Duration: 18 Nov 202320 Nov 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14252 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference25th International Conference on Information and Communications Security, ICICS 2023

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.


  • Deep learning
  • Robustness
  • User classification
  • VR


Dive into the research topics of 'Revisiting the Deep Learning-Based Eavesdropping Attacks via Facial Dynamics from VR Motion Sensors'. Together they form a unique fingerprint.

Cite this