Browse with me: An online clothes shopping assistantfor people with visual impairments

Abigale J. Stangl; Esha Kothari; Suyog D. Jain; Tom Yeh; Kristen Grauman; Danna Gurari

doi:10.1145/3234695.3236337

Browse with me: An online clothes shopping assistantfor people with visual impairments

Abigale J. Stangl, Esha Kothari, Suyog D. Jain, Tom Yeh, Kristen Grauman, Danna Gurari

Department of Psychology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

39 Scopus citations

Abstract

Our interviewswithpeoplewhohave visual impairmentsshow clothes shopping is an important activity in their lives. Unfortunately, clothes shopping web sites remain largely inaccessible. We propose design recommendations to address online accessibility issues reported by visually impaired study participants and an implementation, which we call BrowseWithMe, to address these issues. BrowseWithMe employs artificial intelligence to automatically convert a product web page into a structured representation that enables a user to interactively ask the BrowseWithMe system what the user wants to learn abouta product (e.g., Whatis the price? Can I see a magnified image of the pants?). This enables people to be active solicitors of the specific information they are seeking rather than passivelisteners of unparsed information. Experiments demonstrate BrowseWithMe can make online clothes shopping more accessible and produce accurate image descriptions.

Original language	English
Title of host publication	ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility
Publisher	Association for Computing Machinery, Inc
Pages	107-118
Number of pages	12
ISBN (Electronic)	9781450356503
DOIs	https://doi.org/10.1145/3234695.3236337
State	Published - 8 Oct 2018
Event	20th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2018 - Galway, Ireland Duration: 22 Oct 2018 → 24 Oct 2018

Publication series

Name	ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility

Conference

Conference	20th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2018
Country/Territory	Ireland
City	Galway
Period	22/10/18 → 24/10/18

Bibliographical note

Funding Information:
We thank study participants and crowd workers for their valuable time. This work is supported by NSF awards to D. G. (IIS-1755593) and K. G. (IIS-1514118) aswellasan Amazon ResearchAwardtoK.G.

Publisher Copyright:
© 2018 Copyright heldby theowner/author(s).

Access to Document

10.1145/3234695.3236337

Cite this

Stangl, A. J., Kothari, E., Jain, S. D., Yeh, T., Grauman, K., & Gurari, D. (2018). Browse with me: An online clothes shopping assistantfor people with visual impairments. In ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 107-118). (ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility). Association for Computing Machinery, Inc. https://doi.org/10.1145/3234695.3236337

Stangl, Abigale J. ; Kothari, Esha ; Jain, Suyog D. et al. / Browse with me : An online clothes shopping assistantfor people with visual impairments. ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. Association for Computing Machinery, Inc, 2018. pp. 107-118 (ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility).

@inproceedings{1e0b5bb601cd40d598c7c99e6064e4f5,

title = "Browse with me: An online clothes shopping assistantfor people with visual impairments",

abstract = "Our interviewswithpeoplewhohave visual impairmentsshow clothes shopping is an important activity in their lives. Unfortunately, clothes shopping web sites remain largely inaccessible. We propose design recommendations to address online accessibility issues reported by visually impaired study participants and an implementation, which we call BrowseWithMe, to address these issues. BrowseWithMe employs artificial intelligence to automatically convert a product web page into a structured representation that enables a user to interactively ask the BrowseWithMe system what the user wants to learn abouta product (e.g., Whatis the price? Can I see a magnified image of the pants?). This enables people to be active solicitors of the specific information they are seeking rather than passivelisteners of unparsed information. Experiments demonstrate BrowseWithMe can make online clothes shopping more accessible and produce accurate image descriptions.",

author = "Stangl, {Abigale J.} and Esha Kothari and Jain, {Suyog D.} and Tom Yeh and Kristen Grauman and Danna Gurari",

note = "Funding Information: We thank study participants and crowd workers for their valuable time. This work is supported by NSF awards to D. G. (IIS-1755593) and K. G. (IIS-1514118) aswellasan Amazon ResearchAwardtoK.G. Publisher Copyright: {\textcopyright} 2018 Copyright heldby theowner/author(s).; 20th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2018 ; Conference date: 22-10-2018 Through 24-10-2018",

year = "2018",

month = oct,

day = "8",

doi = "10.1145/3234695.3236337",

language = "English",

series = "ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility",

publisher = "Association for Computing Machinery, Inc",

pages = "107--118",

booktitle = "ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility",

}

Stangl, AJ, Kothari, E, Jain, SD, Yeh, T, Grauman, K & Gurari, D 2018, Browse with me: An online clothes shopping assistantfor people with visual impairments. in ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, Association for Computing Machinery, Inc, pp. 107-118, 20th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2018, Galway, Ireland, 22/10/18. https://doi.org/10.1145/3234695.3236337

Browse with me: An online clothes shopping assistantfor people with visual impairments. / Stangl, Abigale J.; Kothari, Esha; Jain, Suyog D. et al.
ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. Association for Computing Machinery, Inc, 2018. p. 107-118 (ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Browse with me

T2 - 20th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2018

AU - Stangl, Abigale J.

AU - Kothari, Esha

AU - Jain, Suyog D.

AU - Yeh, Tom

AU - Grauman, Kristen

AU - Gurari, Danna

N1 - Funding Information: We thank study participants and crowd workers for their valuable time. This work is supported by NSF awards to D. G. (IIS-1755593) and K. G. (IIS-1514118) aswellasan Amazon ResearchAwardtoK.G. Publisher Copyright: © 2018 Copyright heldby theowner/author(s).

PY - 2018/10/8

Y1 - 2018/10/8

N2 - Our interviewswithpeoplewhohave visual impairmentsshow clothes shopping is an important activity in their lives. Unfortunately, clothes shopping web sites remain largely inaccessible. We propose design recommendations to address online accessibility issues reported by visually impaired study participants and an implementation, which we call BrowseWithMe, to address these issues. BrowseWithMe employs artificial intelligence to automatically convert a product web page into a structured representation that enables a user to interactively ask the BrowseWithMe system what the user wants to learn abouta product (e.g., Whatis the price? Can I see a magnified image of the pants?). This enables people to be active solicitors of the specific information they are seeking rather than passivelisteners of unparsed information. Experiments demonstrate BrowseWithMe can make online clothes shopping more accessible and produce accurate image descriptions.

AB - Our interviewswithpeoplewhohave visual impairmentsshow clothes shopping is an important activity in their lives. Unfortunately, clothes shopping web sites remain largely inaccessible. We propose design recommendations to address online accessibility issues reported by visually impaired study participants and an implementation, which we call BrowseWithMe, to address these issues. BrowseWithMe employs artificial intelligence to automatically convert a product web page into a structured representation that enables a user to interactively ask the BrowseWithMe system what the user wants to learn abouta product (e.g., Whatis the price? Can I see a magnified image of the pants?). This enables people to be active solicitors of the specific information they are seeking rather than passivelisteners of unparsed information. Experiments demonstrate BrowseWithMe can make online clothes shopping more accessible and produce accurate image descriptions.

UR - http://www.scopus.com/inward/record.url?scp=85056856429&partnerID=8YFLogxK

U2 - 10.1145/3234695.3236337

DO - 10.1145/3234695.3236337

M3 - Conference contribution

AN - SCOPUS:85056856429

T3 - ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility

SP - 107

EP - 118

BT - ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility

PB - Association for Computing Machinery, Inc

Y2 - 22 October 2018 through 24 October 2018

ER -

Stangl AJ, Kothari E, Jain SD, Yeh T, Grauman K, Gurari D. Browse with me: An online clothes shopping assistantfor people with visual impairments. In ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. Association for Computing Machinery, Inc. 2018. p. 107-118. (ASSETS 2018 - Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility). doi: 10.1145/3234695.3236337

Browse with me: An online clothes shopping assistantfor people with visual impairments

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Fingerprint

Cite this