Reactile: Programming swarm user interfaces through direct physical manipulation

Ryo Suzuki; Jun Kato; Mark D. Gross; Tom Yeh

doi:10.1145/3173574.3173773

Reactile: Programming swarm user interfaces through direct physical manipulation

Ryo Suzuki, Jun Kato, Mark D. Gross, Tom Yeh

Department of Psychology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

32 Scopus citations

Abstract

We explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of low-level controls. In contrast, our approach allows programmers to work in physical space by directly manipulating objects and think in terms of highlevel interface design. Inspired by current UI programming practices, we introduce a four-step workflow-create elements, abstract attributes, specify behaviors, and propagate changes- for Swarm UI programming. We propose a set of direct physical manipulation techniques to support each step in this workflow. To demonstrate these concepts, we developed Reactile, a Swarm UI programming environment that actuates a swarm of small magnets and displays spatial information of program states using a DLP projector. Two user studies-an in-class survey with 148 students and a lab interview with eight participants-confirm that our approach is intuitive and understandable for programming Swarm UIs.

Original language	English
Title of host publication	CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
Subtitle of host publication	Engage with CHI
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450356206, 9781450356213
DOIs	https://doi.org/10.1145/3173574.3173773
State	Published - 20 Apr 2018
Event	2018 CHI Conference on Human Factors in Computing Systems, CHI 2018 - Montreal, Canada Duration: 21 Apr 2018 → 26 Apr 2018

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings
Volume	2018-April

Conference

Conference	2018 CHI Conference on Human Factors in Computing Systems, CHI 2018
Country/Territory	Canada
City	Montreal
Period	21/04/18 → 26/04/18

Bibliographical note

Publisher Copyright:
© 2018 Copyright is held by the owner/author(s).

Keywords

Direct manipulation
Programming by demonstration
Swarm user interfaces
Tangible programming

Access to Document

10.1145/3173574.3173773

Cite this

Suzuki, R., Kato, J., Gross, M. D., & Yeh, T. (2018). Reactile: Programming swarm user interfaces through direct physical manipulation. In CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI (Conference on Human Factors in Computing Systems - Proceedings; Vol. 2018-April). Association for Computing Machinery. https://doi.org/10.1145/3173574.3173773

@inproceedings{5df3d9a783824673b463bed49147f256,

title = "Reactile: Programming swarm user interfaces through direct physical manipulation",

abstract = "We explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of low-level controls. In contrast, our approach allows programmers to work in physical space by directly manipulating objects and think in terms of highlevel interface design. Inspired by current UI programming practices, we introduce a four-step workflow-create elements, abstract attributes, specify behaviors, and propagate changes- for Swarm UI programming. We propose a set of direct physical manipulation techniques to support each step in this workflow. To demonstrate these concepts, we developed Reactile, a Swarm UI programming environment that actuates a swarm of small magnets and displays spatial information of program states using a DLP projector. Two user studies-an in-class survey with 148 students and a lab interview with eight participants-confirm that our approach is intuitive and understandable for programming Swarm UIs.",

keywords = "Direct manipulation, Programming by demonstration, Swarm user interfaces, Tangible programming",

author = "Ryo Suzuki and Jun Kato and Gross, {Mark D.} and Tom Yeh",

note = "Publisher Copyright: {\textcopyright} 2018 Copyright is held by the owner/author(s).; 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018 ; Conference date: 21-04-2018 Through 26-04-2018",

year = "2018",

month = apr,

day = "20",

doi = "10.1145/3173574.3173773",

language = "English",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

booktitle = "CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems",

}

Suzuki, R, Kato, J, Gross, MD & Yeh, T 2018, Reactile: Programming swarm user interfaces through direct physical manipulation. in CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. Conference on Human Factors in Computing Systems - Proceedings, vol. 2018-April, Association for Computing Machinery, 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, Montreal, Canada, 21/04/18. https://doi.org/10.1145/3173574.3173773

Reactile: Programming swarm user interfaces through direct physical manipulation. / Suzuki, Ryo; Kato, Jun; Gross, Mark D. et al.
CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. Association for Computing Machinery, 2018. (Conference on Human Factors in Computing Systems - Proceedings; Vol. 2018-April).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reactile

T2 - 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018

AU - Suzuki, Ryo

AU - Kato, Jun

AU - Gross, Mark D.

AU - Yeh, Tom

PY - 2018/4/20

Y1 - 2018/4/20

N2 - We explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of low-level controls. In contrast, our approach allows programmers to work in physical space by directly manipulating objects and think in terms of highlevel interface design. Inspired by current UI programming practices, we introduce a four-step workflow-create elements, abstract attributes, specify behaviors, and propagate changes- for Swarm UI programming. We propose a set of direct physical manipulation techniques to support each step in this workflow. To demonstrate these concepts, we developed Reactile, a Swarm UI programming environment that actuates a swarm of small magnets and displays spatial information of program states using a DLP projector. Two user studies-an in-class survey with 148 students and a lab interview with eight participants-confirm that our approach is intuitive and understandable for programming Swarm UIs.

AB - We explore a new approach to programming swarm user interfaces (Swarm UI) by leveraging direct physical manipulation. Existing Swarm UI applications are written using a robot programming framework: users work on a computer screen and think in terms of low-level controls. In contrast, our approach allows programmers to work in physical space by directly manipulating objects and think in terms of highlevel interface design. Inspired by current UI programming practices, we introduce a four-step workflow-create elements, abstract attributes, specify behaviors, and propagate changes- for Swarm UI programming. We propose a set of direct physical manipulation techniques to support each step in this workflow. To demonstrate these concepts, we developed Reactile, a Swarm UI programming environment that actuates a swarm of small magnets and displays spatial information of program states using a DLP projector. Two user studies-an in-class survey with 148 students and a lab interview with eight participants-confirm that our approach is intuitive and understandable for programming Swarm UIs.

KW - Direct manipulation

KW - Programming by demonstration

KW - Swarm user interfaces

KW - Tangible programming

UR - http://www.scopus.com/inward/record.url?scp=85046966675&partnerID=8YFLogxK

U2 - 10.1145/3173574.3173773

DO - 10.1145/3173574.3173773

M3 - Conference contribution

AN - SCOPUS:85046966675

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

Y2 - 21 April 2018 through 26 April 2018

ER -

Suzuki R, Kato J, Gross MD, Yeh T. Reactile: Programming swarm user interfaces through direct physical manipulation. In CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. Association for Computing Machinery. 2018. (Conference on Human Factors in Computing Systems - Proceedings). doi: 10.1145/3173574.3173773

Reactile: Programming swarm user interfaces through direct physical manipulation

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this