Say and find it: A multimodal wearable interface for people with visual impairment

Taeyong Kim, Sanghong Kim, Joonhee Choi, Youngsun Lee, Bowon Lee

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Recent advances in computer vision and natural language processing using deep neural networks (DNNs) have enabled rich and intuitive multimodal interfaces. However, research on intelligent assistance systems for persons with visual impairment has not been well explored. In this work, we present an interactive object recognition and guidance interface based on multimodal interaction for blind and partially sighted people using an embedded mobile device. We demonstrate that the proposed solution using DNNs can effectively assist visually impaired people. We believe that this work will provide new and helpful insights for designing intelligent assistance systems in the future.

Original languageEnglish
Title of host publicationUIST 2019 Adjunct - Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology
PublisherAssociation for Computing Machinery, Inc
Pages27-29
Number of pages3
ISBN (Electronic)9781450368179
DOIs
StatePublished - 14 Oct 2019
Event32nd Annual ACM Symposium on User Interface Software and Technology, UIST 2019 - New Orleans, United States
Duration: 20 Oct 201923 Oct 2019

Publication series

NameUIST 2019 Adjunct - Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology

Conference

Conference32nd Annual ACM Symposium on User Interface Software and Technology, UIST 2019
Country/TerritoryUnited States
CityNew Orleans
Period20/10/1923/10/19

Bibliographical note

Funding Information:
This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2018S1A5A2A03037308) and by the Industrial Technology Innovation Program funded by the Ministry of Trade, Industry & Energy (10073154).

Publisher Copyright:
© 2019 Copyright is held by the owner/author(s).

Keywords

  • Assistive system
  • Mobile interface
  • Multimodal wearable interface
  • Visual impairment

Fingerprint

Dive into the research topics of 'Say and find it: A multimodal wearable interface for people with visual impairment'. Together they form a unique fingerprint.

Cite this