The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind

Lee Stearns; Ruofei Du; Uran Oh; Yumeng Wang; Leah Findlater; Rama Chellappa; Jon E. Froehlich

doi:10.1007/978-3-319-16199-0_43

The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind

Lee Stearns, Ruofei Du, Uran Oh, Yumeng Wang, Leah Findlater, Rama Chellappa, Jon E. Froehlich

Department of Computer Science & Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Scopus citations

Abstract

We introduce the preliminary design of a novel vision-augmented touch system called HandSight intended to support activities of daily living (ADLs) by sensing and feeding back non-tactile information about the physical world as it is touched. Though we are interested in supporting a range of ADL applications, here we focus specifically on reading printed text. We discuss our vision for HandSight, describe its current implementation and results from an initial performance analysis of finger-based text scanning. We then present a user study with four visually impaired participants (three blind) exploring how to continuously guide a user’s finger across text using three feedback conditions (haptic, audio, and both). Though preliminary, our results show that participants valued the ability to access printed material, and that, in contrast to previous findings, audio finger guidance may result in the best reading performance.

Original language	English
Title of host publication	Computer Vision - ECCV 2014 Workshops, Proceedings
Editors	Carsten Rother, Lourdes Agapito, Michael M. Bronstein
Publisher	Springer Verlag
Pages	615-631
Number of pages	17
ISBN (Electronic)	9783319161983
DOIs	https://doi.org/10.1007/978-3-319-16199-0_43
State	Published - 2015
Event	13th European Conference on Computer Vision, ECCV 2014 - Zurich, Switzerland Duration: 6 Sep 2014 → 12 Sep 2014

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8927
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	13th European Conference on Computer Vision, ECCV 2014
Country/Territory	Switzerland
City	Zurich
Period	6/09/14 → 12/09/14

Bibliographical note

Publisher Copyright:
© Springer International Publishing Switzerland 2015.

Keywords

Accessibility
Real-time OCR
Text reading for blind
Wearables

Access to Document

10.1007/978-3-319-16199-0_43

Cite this

Stearns, L., Du, R., Oh, U., Wang, Y., Findlater, L., Chellappa, R., & Froehlich, J. E. (2015). The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind. In C. Rother, L. Agapito, & M. M. Bronstein (Eds.), Computer Vision - ECCV 2014 Workshops, Proceedings (pp. 615-631). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8927). Springer Verlag. https://doi.org/10.1007/978-3-319-16199-0_43

Stearns, Lee ; Du, Ruofei ; Oh, Uran et al. / The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind. Computer Vision - ECCV 2014 Workshops, Proceedings. editor / Carsten Rother ; Lourdes Agapito ; Michael M. Bronstein. Springer Verlag, 2015. pp. 615-631 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9560e02503084041bfd8ef0c40e91e53,

title = "The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind",

abstract = "We introduce the preliminary design of a novel vision-augmented touch system called HandSight intended to support activities of daily living (ADLs) by sensing and feeding back non-tactile information about the physical world as it is touched. Though we are interested in supporting a range of ADL applications, here we focus specifically on reading printed text. We discuss our vision for HandSight, describe its current implementation and results from an initial performance analysis of finger-based text scanning. We then present a user study with four visually impaired participants (three blind) exploring how to continuously guide a user{\textquoteright}s finger across text using three feedback conditions (haptic, audio, and both). Though preliminary, our results show that participants valued the ability to access printed material, and that, in contrast to previous findings, audio finger guidance may result in the best reading performance.",

keywords = "Accessibility, Real-time OCR, Text reading for blind, Wearables",

author = "Lee Stearns and Ruofei Du and Uran Oh and Yumeng Wang and Leah Findlater and Rama Chellappa and Froehlich, {Jon E.}",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 13th European Conference on Computer Vision, ECCV 2014 ; Conference date: 06-09-2014 Through 12-09-2014",

year = "2015",

doi = "10.1007/978-3-319-16199-0_43",

language = "English",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "615--631",

editor = "Carsten Rother and Lourdes Agapito and Bronstein, {Michael M.}",

booktitle = "Computer Vision - ECCV 2014 Workshops, Proceedings",

}

Stearns, L, Du, R, Oh, U, Wang, Y, Findlater, L, Chellappa, R & Froehlich, JE 2015, The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind. in C Rother, L Agapito & MM Bronstein (eds), Computer Vision - ECCV 2014 Workshops, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8927, Springer Verlag, pp. 615-631, 13th European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland, 6/09/14. https://doi.org/10.1007/978-3-319-16199-0_43

The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind. / Stearns, Lee; Du, Ruofei; Oh, Uran et al.
Computer Vision - ECCV 2014 Workshops, Proceedings. ed. / Carsten Rother; Lourdes Agapito; Michael M. Bronstein. Springer Verlag, 2015. p. 615-631 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8927).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind

AU - Stearns, Lee

AU - Du, Ruofei

AU - Oh, Uran

AU - Wang, Yumeng

AU - Findlater, Leah

AU - Chellappa, Rama

AU - Froehlich, Jon E.

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2015.

PY - 2015

Y1 - 2015

N2 - We introduce the preliminary design of a novel vision-augmented touch system called HandSight intended to support activities of daily living (ADLs) by sensing and feeding back non-tactile information about the physical world as it is touched. Though we are interested in supporting a range of ADL applications, here we focus specifically on reading printed text. We discuss our vision for HandSight, describe its current implementation and results from an initial performance analysis of finger-based text scanning. We then present a user study with four visually impaired participants (three blind) exploring how to continuously guide a user’s finger across text using three feedback conditions (haptic, audio, and both). Though preliminary, our results show that participants valued the ability to access printed material, and that, in contrast to previous findings, audio finger guidance may result in the best reading performance.

AB - We introduce the preliminary design of a novel vision-augmented touch system called HandSight intended to support activities of daily living (ADLs) by sensing and feeding back non-tactile information about the physical world as it is touched. Though we are interested in supporting a range of ADL applications, here we focus specifically on reading printed text. We discuss our vision for HandSight, describe its current implementation and results from an initial performance analysis of finger-based text scanning. We then present a user study with four visually impaired participants (three blind) exploring how to continuously guide a user’s finger across text using three feedback conditions (haptic, audio, and both). Though preliminary, our results show that participants valued the ability to access printed material, and that, in contrast to previous findings, audio finger guidance may result in the best reading performance.

KW - Accessibility

KW - Real-time OCR

KW - Text reading for blind

KW - Wearables

UR - http://www.scopus.com/inward/record.url?scp=84928782809&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-16199-0_43

DO - 10.1007/978-3-319-16199-0_43

M3 - Conference contribution

AN - SCOPUS:84928782809

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 615

EP - 631

BT - Computer Vision - ECCV 2014 Workshops, Proceedings

A2 - Rother, Carsten

A2 - Agapito, Lourdes

A2 - Bronstein, Michael M.

PB - Springer Verlag

T2 - 13th European Conference on Computer Vision, ECCV 2014

Y2 - 6 September 2014 through 12 September 2014

ER -

Stearns L, Du R, Oh U, Wang Y, Findlater L, Chellappa R et al. The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind. In Rother C, Agapito L, Bronstein MM, editors, Computer Vision - ECCV 2014 Workshops, Proceedings. Springer Verlag. 2015. p. 615-631. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-16199-0_43

The design and preliminary evaluation of a finger-mounted camera and feedback system to enable reading of printed text for the blind

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this