Follow that sound: Using sonification and corrective verbal feedback to teach touchscreen gestures

Uran Oh, Shaun K. Kane, Leah Findlater

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

53 Scopus citations

Abstract

While sighted users may learn to perform touchscreen gestures through observation (e.g., of other users or video tutorials), such mechanisms are inaccessible for users with visual impairments. As a result, learning to perform gestures can be challenging. We propose and evaluate two techniques to teach touchscreen gestures to users with visual impairments: (1) corrective verbal feedback using text-to-speech and automatic analysis of the user's drawn gesture; (2) gesture sonification to generate sound based on finger touches, creating an audio representation of a gesture. To refine and evaluate the techniques, we conducted two controlled lab studies. The first study, with 12 sighted participants, compared parameters for sonifying gestures in an eyes-free scenario and identified pitch + stereo panning as the best combination. In the second study, 6 blind and low-vision participants completed gesture replication tasks with the two feedback techniques. Subjective data and preliminary performance findings indicate that the techniques offer complementary advantages.

Original languageEnglish
Title of host publicationProceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2013
DOIs
StatePublished - 2013
Event15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2013 - Bellevue, WA, United States
Duration: 21 Oct 201323 Oct 2013

Publication series

NameProceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2013

Conference

Conference15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2013
Country/TerritoryUnited States
CityBellevue, WA
Period21/10/1323/10/13

Keywords

  • Blindness
  • Gestures
  • Sonification
  • Touchscreen
  • Visual impairments

Fingerprint

Dive into the research topics of 'Follow that sound: Using sonification and corrective verbal feedback to teach touchscreen gestures'. Together they form a unique fingerprint.

Cite this