Due to its simplicity and efficiency for evaluating product attributes, check-all-that-apply (CATA) questions have been widely used. Yet, CATA questions have been reported to lack discriminability of subtle product differences and suffer from the problems of response bias from satisficing and acquiescence behaviors. In the present paper, a novel two-step rating-based ‘double-faced applicability’ test was developed as an extended response format of CATA, to improve its sensitivity for product discrimination and for stabilizing subjects’ evaluative criteria. In the ‘double-faced applicability’ test, each attribute was ‘double-faced’, meaning that two descriptors (a pair of semantic-differential descriptors) are separately presented in the questionnaires representing both sides of each attribute. For performing the two-step rating on each attribute in the questionnaire, subjects are instructed to first respond ‘Yes (does apply)’ or ‘No (does not apply)’ and then to answer a 3-point sureness rating (how sure they were about their Yes or No response). The performance of the two-step ratings in this new method was compared to a simple one-step applicability rating test method as well as the forced-choice Yes/No questions without the sureness rating in terms of sensitivity in sample discrimination. The results showed that the ‘double-faced applicability’ test provided better product discrimination and showed the potential to reduce acquiescence response bias when using existing variants of CATA response formats.
Bibliographical noteFunding Information:
This research was supported by Unilever R&D and the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2015R1A1A1A05001170 ). The authors also thank Emma Elliott at Unilever R&D Port Sunlight for preparing the samples, Hyun-Kyung Shin, Yu-Na Jeong, Ji-Young Yoon, Hye-Jong Yoo, Bi-A Kang and So-Yub Lee at Ewha Womans University for their assistance in conducting experiments, and Timo Giesbrecht at Unilever for reviewing and improving the manuscript.
© 2016 Elsevier Ltd
- Applicability scores
- Intensity scales
- Product discrimination
- Signal detection theory
- Usage experience