Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation

Hattan Althebeiti; Ran Gedawy; Ahod Alghuried; Daehun Nyang; David Mohaisen

doi:10.1007/978-981-99-8024-6_13

Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation

Hattan Althebeiti, Ran Gedawy, Ahod Alghuried, Daehun Nyang, David Mohaisen

Department of Cyber Security

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Augmented reality (AR) interaction methods are leaning towards more natural techniques, such as voice commands, hand gestures, and in-air tapping for input. From a security perspective, however, recent works have demonstrated that these methods, such as in-air tapping, are vulnerable to inference attacks where an adversary is capable of reconstructing input in the virtual environment using low-level hand-tracking data with high accuracy. This paper addresses the defense of in-air tapping mechanisms against inference attacks by developing and evaluating a 3D curved keyboard for input. Our design exploits the symmetry between the virtual and physical worlds enabling the inference attack in the first place and increasing the uncertainty of the adversary by manipulating the geometric aspects of this keyboard plane in 3D. We evaluate our design through numerous experiments and show it to be robust against inference attacks, where the adversary’s accuracy in obtaining the correct input text is reduced to 0% (from 87%) and at most to just 18% within the top-500 candidate reconstructions.

Original language	English
Title of host publication	Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers
Editors	Howon Kim, Jonghee Youn
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	159-174
Number of pages	16
ISBN (Print)	9789819980239
DOIs	https://doi.org/10.1007/978-981-99-8024-6_13
State	Published - 2024
Event	24th International Conference on Information Security Applications, WISA 2023 - Jeju Island, Korea, Republic of Duration: 23 Aug 2023 → 25 Aug 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14402 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Information Security Applications, WISA 2023
Country/Territory	Korea, Republic of
City	Jeju Island
Period	23/08/23 → 25/08/23

Bibliographical note

Publisher Copyright:
© 2024, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Keywords

AR/VR
Defense
Inference
Privacy

Access to Document

10.1007/978-981-99-8024-6_13

Cite this

Althebeiti, H., Gedawy, R., Alghuried, A., Nyang, D., & Mohaisen, D. (2024). Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation. In H. Kim, & J. Youn (Eds.), Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers (pp. 159-174). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14402 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-8024-6_13

Althebeiti, Hattan ; Gedawy, Ran ; Alghuried, Ahod et al. / Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts : Design and Evaluation. Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers. editor / Howon Kim ; Jonghee Youn. Springer Science and Business Media Deutschland GmbH, 2024. pp. 159-174 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{f5f16285cb2f4bb18db854030aa66cbb,

title = "Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation",

abstract = "Augmented reality (AR) interaction methods are leaning towards more natural techniques, such as voice commands, hand gestures, and in-air tapping for input. From a security perspective, however, recent works have demonstrated that these methods, such as in-air tapping, are vulnerable to inference attacks where an adversary is capable of reconstructing input in the virtual environment using low-level hand-tracking data with high accuracy. This paper addresses the defense of in-air tapping mechanisms against inference attacks by developing and evaluating a 3D curved keyboard for input. Our design exploits the symmetry between the virtual and physical worlds enabling the inference attack in the first place and increasing the uncertainty of the adversary by manipulating the geometric aspects of this keyboard plane in 3D. We evaluate our design through numerous experiments and show it to be robust against inference attacks, where the adversary{\textquoteright}s accuracy in obtaining the correct input text is reduced to 0% (from 87%) and at most to just 18% within the top-500 candidate reconstructions.",

keywords = "AR/VR, Defense, Inference, Privacy",

author = "Hattan Althebeiti and Ran Gedawy and Ahod Alghuried and Daehun Nyang and David Mohaisen",

note = "Publisher Copyright: {\textcopyright} 2024, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 24th International Conference on Information Security Applications, WISA 2023 ; Conference date: 23-08-2023 Through 25-08-2023",

year = "2024",

doi = "10.1007/978-981-99-8024-6_13",

language = "English",

isbn = "9789819980239",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "159--174",

editor = "Howon Kim and Jonghee Youn",

booktitle = "Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers",

}

Althebeiti, H, Gedawy, R, Alghuried, A, Nyang, D & Mohaisen, D 2024, Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation. in H Kim & J Youn (eds), Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14402 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 159-174, 24th International Conference on Information Security Applications, WISA 2023, Jeju Island, Korea, Republic of, 23/08/23. https://doi.org/10.1007/978-981-99-8024-6_13

Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation. / Althebeiti, Hattan; Gedawy, Ran; Alghuried, Ahod et al.
Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers. ed. / Howon Kim; Jonghee Youn. Springer Science and Business Media Deutschland GmbH, 2024. p. 159-174 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14402 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts

T2 - 24th International Conference on Information Security Applications, WISA 2023

AU - Althebeiti, Hattan

AU - Gedawy, Ran

AU - Alghuried, Ahod

AU - Nyang, Daehun

AU - Mohaisen, David

PY - 2024

Y1 - 2024

N2 - Augmented reality (AR) interaction methods are leaning towards more natural techniques, such as voice commands, hand gestures, and in-air tapping for input. From a security perspective, however, recent works have demonstrated that these methods, such as in-air tapping, are vulnerable to inference attacks where an adversary is capable of reconstructing input in the virtual environment using low-level hand-tracking data with high accuracy. This paper addresses the defense of in-air tapping mechanisms against inference attacks by developing and evaluating a 3D curved keyboard for input. Our design exploits the symmetry between the virtual and physical worlds enabling the inference attack in the first place and increasing the uncertainty of the adversary by manipulating the geometric aspects of this keyboard plane in 3D. We evaluate our design through numerous experiments and show it to be robust against inference attacks, where the adversary’s accuracy in obtaining the correct input text is reduced to 0% (from 87%) and at most to just 18% within the top-500 candidate reconstructions.

AB - Augmented reality (AR) interaction methods are leaning towards more natural techniques, such as voice commands, hand gestures, and in-air tapping for input. From a security perspective, however, recent works have demonstrated that these methods, such as in-air tapping, are vulnerable to inference attacks where an adversary is capable of reconstructing input in the virtual environment using low-level hand-tracking data with high accuracy. This paper addresses the defense of in-air tapping mechanisms against inference attacks by developing and evaluating a 3D curved keyboard for input. Our design exploits the symmetry between the virtual and physical worlds enabling the inference attack in the first place and increasing the uncertainty of the adversary by manipulating the geometric aspects of this keyboard plane in 3D. We evaluate our design through numerous experiments and show it to be robust against inference attacks, where the adversary’s accuracy in obtaining the correct input text is reduced to 0% (from 87%) and at most to just 18% within the top-500 candidate reconstructions.

KW - AR/VR

KW - Defense

KW - Inference

KW - Privacy

UR - http://www.scopus.com/inward/record.url?scp=85182608677&partnerID=8YFLogxK

U2 - 10.1007/978-981-99-8024-6_13

DO - 10.1007/978-981-99-8024-6_13

M3 - Conference contribution

AN - SCOPUS:85182608677

SN - 9789819980239

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 159

EP - 174

BT - Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers

A2 - Kim, Howon

A2 - Youn, Jonghee

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 August 2023 through 25 August 2023

ER -

Althebeiti H, Gedawy R, Alghuried A, Nyang D, Mohaisen D. Defending AirType Against Inference Attacks Using 3D In-Air Keyboard Layouts: Design and Evaluation. In Kim H, Youn J, editors, Information Security Applications - 24th International Conference, WISA 2023, Jeju Island, South Korea, August 23–25, 2023, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2024. p. 159-174. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-99-8024-6_13