TY - GEN
T1 - GUI testing using computer vision
AU - Chang, Tsung Hsiang
AU - Yeh, Tom
AU - Miller, Robert C.
PY - 2010
Y1 - 2010
N2 - Testing a GUI's visual behavior typically requires human testers to interact with the GUI and to observe whether the expected results of interaction are presented. This paper presents a new approach to GUI testing using computer vision for testers to automate their tasks. Testers can write a visual test script that uses images to specify which GUI components to interact with and what visual feedback to be observed. Testers can also generate visual test scripts by demonstration. By recording both input events and screen images, it is possible to extract the images of components interacted with and the visual feedback seen by the demonstrator, and generate a visual test script automatically. We show that a variety of GUI behavior can be tested using this approach. Also, we show how this approach can facilitate good testing practices such as unit testing, regression testing, and test-driven development.
AB - Testing a GUI's visual behavior typically requires human testers to interact with the GUI and to observe whether the expected results of interaction are presented. This paper presents a new approach to GUI testing using computer vision for testers to automate their tasks. Testers can write a visual test script that uses images to specify which GUI components to interact with and what visual feedback to be observed. Testers can also generate visual test scripts by demonstration. By recording both input events and screen images, it is possible to extract the images of components interacted with and the visual feedback seen by the demonstrator, and generate a visual test script automatically. We show that a variety of GUI behavior can be tested using this approach. Also, we show how this approach can facilitate good testing practices such as unit testing, regression testing, and test-driven development.
KW - gui automation
KW - gui testing
KW - test by demonstration
UR - http://www.scopus.com/inward/record.url?scp=77953968759&partnerID=8YFLogxK
U2 - 10.1145/1753326.1753555
DO - 10.1145/1753326.1753555
M3 - Conference contribution
AN - SCOPUS:77953968759
SN - 9781605589299
T3 - Conference on Human Factors in Computing Systems - Proceedings
SP - 1535
EP - 1544
BT - CHI 2010 - The 28th Annual CHI Conference on Human Factors in Computing Systems, Conference Proceedings
T2 - 28th Annual CHI Conference on Human Factors in Computing Systems, CHI 2010
Y2 - 10 April 2010 through 15 April 2010
ER -