Exploring the design space of an augmented display for conveying facial expressions for people with autism

Seunga Chung; Uran Oh

doi:10.1109/ISMAR-Adjunct.2019.00049

Exploring the design space of an augmented display for conveying facial expressions for people with autism

Seunga Chung, Uran Oh

Department of Computer Science & Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Facial expression is considered as the most intuitive and effective way of conveying one's emotion among other nonverbal interactions.However, people with autism have limited access to this rich communication channel due to their inability to read facial expressions. To help them be aware of others' emotions, we developed a CNN-based facial expression recognition system using Microsoft Hololens and explored three different modes for displaying facial expressions of a conversation partner varying the levels of explicitness. Subjective feedback from a preliminary study with 6 pilot participants suggests that each mode is worth investigating for serving people with various needs and preferences who wish to receive augmented visual hints on others' emotion.

Original language	English
Title of host publication	Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	435-437
Number of pages	3
ISBN (Electronic)	9781728147659
DOIs	https://doi.org/10.1109/ISMAR-Adjunct.2019.00049
State	Published - Oct 2019
Event	18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 - Beijing, China Duration: 14 Oct 2019 → 18 Oct 2019

Publication series

Name	Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

Conference

Conference	18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019
Country/Territory	China
City	Beijing
Period	14/10/19 → 18/10/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

Autism
Emotion-recognition
Facial-expressions
Mixed-reality

Access to Document

10.1109/ISMAR-Adjunct.2019.00049

Cite this

Chung, S., & Oh, U. (2019). Exploring the design space of an augmented display for conveying facial expressions for people with autism. In Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 (pp. 435-437). Article 8951991 (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR-Adjunct.2019.00049

Chung, Seunga ; Oh, Uran. / Exploring the design space of an augmented display for conveying facial expressions for people with autism. Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 435-437 (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019).

@inproceedings{7c6703e87eea4be0b57af19ea5393067,

title = "Exploring the design space of an augmented display for conveying facial expressions for people with autism",

abstract = "Facial expression is considered as the most intuitive and effective way of conveying one's emotion among other nonverbal interactions.However, people with autism have limited access to this rich communication channel due to their inability to read facial expressions. To help them be aware of others' emotions, we developed a CNN-based facial expression recognition system using Microsoft Hololens and explored three different modes for displaying facial expressions of a conversation partner varying the levels of explicitness. Subjective feedback from a preliminary study with 6 pilot participants suggests that each mode is worth investigating for serving people with various needs and preferences who wish to receive augmented visual hints on others' emotion.",

keywords = "Autism, Emotion-recognition, Facial-expressions, Mixed-reality",

author = "Seunga Chung and Uran Oh",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 ; Conference date: 14-10-2019 Through 18-10-2019",

year = "2019",

month = oct,

doi = "10.1109/ISMAR-Adjunct.2019.00049",

language = "English",

series = "Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "435--437",

booktitle = "Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019",

}

Chung, S & Oh, U 2019, Exploring the design space of an augmented display for conveying facial expressions for people with autism. in Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019., 8951991, Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019, Institute of Electrical and Electronics Engineers Inc., pp. 435-437, 18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019, Beijing, China, 14/10/19. https://doi.org/10.1109/ISMAR-Adjunct.2019.00049

Exploring the design space of an augmented display for conveying facial expressions for people with autism. / Chung, Seunga; Oh, Uran.
Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 435-437 8951991 (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Exploring the design space of an augmented display for conveying facial expressions for people with autism

AU - Chung, Seunga

AU - Oh, Uran

PY - 2019/10

Y1 - 2019/10

N2 - Facial expression is considered as the most intuitive and effective way of conveying one's emotion among other nonverbal interactions.However, people with autism have limited access to this rich communication channel due to their inability to read facial expressions. To help them be aware of others' emotions, we developed a CNN-based facial expression recognition system using Microsoft Hololens and explored three different modes for displaying facial expressions of a conversation partner varying the levels of explicitness. Subjective feedback from a preliminary study with 6 pilot participants suggests that each mode is worth investigating for serving people with various needs and preferences who wish to receive augmented visual hints on others' emotion.

AB - Facial expression is considered as the most intuitive and effective way of conveying one's emotion among other nonverbal interactions.However, people with autism have limited access to this rich communication channel due to their inability to read facial expressions. To help them be aware of others' emotions, we developed a CNN-based facial expression recognition system using Microsoft Hololens and explored three different modes for displaying facial expressions of a conversation partner varying the levels of explicitness. Subjective feedback from a preliminary study with 6 pilot participants suggests that each mode is worth investigating for serving people with various needs and preferences who wish to receive augmented visual hints on others' emotion.

KW - Autism

KW - Emotion-recognition

KW - Facial-expressions

KW - Mixed-reality

UR - http://www.scopus.com/inward/record.url?scp=85078757109&partnerID=8YFLogxK

U2 - 10.1109/ISMAR-Adjunct.2019.00049

DO - 10.1109/ISMAR-Adjunct.2019.00049

M3 - Conference contribution

AN - SCOPUS:85078757109

T3 - Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

SP - 435

EP - 437

BT - Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

Y2 - 14 October 2019 through 18 October 2019

ER -

Chung S, Oh U. Exploring the design space of an augmented display for conveying facial expressions for people with autism. In Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 435-437. 8951991. (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019). doi: 10.1109/ISMAR-Adjunct.2019.00049

Exploring the design space of an augmented display for conveying facial expressions for people with autism

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this