Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore

Kristi Yoonsup Lee; Saudamini Vishwanath Dabak; Vivian Hanxiao Kong; Minah Park; Shirley L.L. Kwok; Madison Silzle; Chayapat Rachatan; Alex Cook; Aly Passanante; Ed Pertwee; Zhengdong Wu; Javier A. Elkin; Heidi J. Larson; Eric H.Y. Lau; Kathy Leung; Joseph T. Wu; Leesa Lin

doi:10.1038/s41746-023-00843-6

Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore

Kristi Yoonsup Lee, Saudamini Vishwanath Dabak, Vivian Hanxiao Kong, Minah Park, Shirley L.L. Kwok, Madison Silzle, Chayapat Rachatan, Alex Cook, Aly Passanante, Ed Pertwee, Zhengdong Wu, Javier A. Elkin, Heidi J. Larson, Eric H.Y. Lau, Kathy Leung, Joseph T. Wu, Leesa Lin

Department of Health Convergence

Research output: Contribution to journal › Article › peer-review

11 Scopus citations

Abstract

Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots’ significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

Original language	English
Article number	96
Journal	npj Digital Medicine
Volume	6
Issue number	1
DOIs	https://doi.org/10.1038/s41746-023-00843-6
State	Published - Dec 2023

Bibliographical note

Funding Information:
This work was supported by the Vaccine Confidence Fund (#VCF-020) and AIR@InnoHK administered by Innovation and Technology Commission. The Health Intervention and Technology Assessment Program (HITAP) was supported by a grant from the Access and Delivery Partnership (ADP), hosted by the United Nations Development Programme (UNDP) and funded by the Government of Japan. We thank Supatnuj Sorndamrih, Wattaporn Thanomsing, Nicha Krishnamra and Sindh R from the Thai Health Promotion Foundation (ThaiHealth) for their support on using the chatbot in Thailand; Dr. Yot Teerawattananon, Assoc. Prof. Wanrudee Isaranuwatchai, Dr. Pritaporn Kingkaew and Ms. Benjarin Santatiwongchai from the Health Intervention and Technology Assessment Program (HITAP) for their inputs on the study; Ruobing Mei from the Laboratory of Data Discovery for Health (D24H) for her input on the study.

Funding Information:
This work was supported by the Vaccine Confidence Fund (#VCF-020) and AIR@InnoHK administered by Innovation and Technology Commission. The Health Intervention and Technology Assessment Program (HITAP) was supported by a grant from the Access and Delivery Partnership (ADP), hosted by the United Nations Development Programme (UNDP) and funded by the Government of Japan. We thank Supatnuj Sorndamrih, Wattaporn Thanomsing, Nicha Krishnamra and Sindh R from the Thai Health Promotion Foundation (ThaiHealth) for their support on using the chatbot in Thailand; Dr. Yot Teerawattananon, Assoc. Prof. Wanrudee Isaranuwatchai, Dr. Pritaporn Kingkaew and Ms. Benjarin Santatiwongchai from the Health Intervention and Technology Assessment Program (HITAP) for their inputs on the study; Ruobing Mei from the Laboratory of Data Discovery for Health (D4H) for her input on the study. 2

Publisher Copyright:
© 2023, The Author(s).

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1038/s41746-023-00843-6

Cite this

Lee, K. Y., Dabak, S. V., Kong, V. H., Park, M., Kwok, S. L. L., Silzle, M., Rachatan, C., Cook, A., Passanante, A., Pertwee, E., Wu, Z., Elkin, J. A., Larson, H. J., Lau, E. H. Y., Leung, K., Wu, J. T., & Lin, L. (2023). Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore. npj Digital Medicine, 6(1), Article 96. https://doi.org/10.1038/s41746-023-00843-6

@article{6eeb950cd4444884b27eded9c80a77a0,

title = "Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore",

abstract = "Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots{\textquoteright} significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.",

author = "Lee, {Kristi Yoonsup} and Dabak, {Saudamini Vishwanath} and Kong, {Vivian Hanxiao} and Minah Park and Kwok, {Shirley L.L.} and Madison Silzle and Chayapat Rachatan and Alex Cook and Aly Passanante and Ed Pertwee and Zhengdong Wu and Elkin, {Javier A.} and Larson, {Heidi J.} and Lau, {Eric H.Y.} and Kathy Leung and Wu, {Joseph T.} and Leesa Lin",

note = "Funding Information: This work was supported by the Vaccine Confidence Fund (#VCF-020) and AIR@InnoHK administered by Innovation and Technology Commission. The Health Intervention and Technology Assessment Program (HITAP) was supported by a grant from the Access and Delivery Partnership (ADP), hosted by the United Nations Development Programme (UNDP) and funded by the Government of Japan. We thank Supatnuj Sorndamrih, Wattaporn Thanomsing, Nicha Krishnamra and Sindh R from the Thai Health Promotion Foundation (ThaiHealth) for their support on using the chatbot in Thailand; Dr. Yot Teerawattananon, Assoc. Prof. Wanrudee Isaranuwatchai, Dr. Pritaporn Kingkaew and Ms. Benjarin Santatiwongchai from the Health Intervention and Technology Assessment Program (HITAP) for their inputs on the study; Ruobing Mei from the Laboratory of Data Discovery for Health (D24H) for her input on the study. Funding Information: This work was supported by the Vaccine Confidence Fund (#VCF-020) and AIR@InnoHK administered by Innovation and Technology Commission. The Health Intervention and Technology Assessment Program (HITAP) was supported by a grant from the Access and Delivery Partnership (ADP), hosted by the United Nations Development Programme (UNDP) and funded by the Government of Japan. We thank Supatnuj Sorndamrih, Wattaporn Thanomsing, Nicha Krishnamra and Sindh R from the Thai Health Promotion Foundation (ThaiHealth) for their support on using the chatbot in Thailand; Dr. Yot Teerawattananon, Assoc. Prof. Wanrudee Isaranuwatchai, Dr. Pritaporn Kingkaew and Ms. Benjarin Santatiwongchai from the Health Intervention and Technology Assessment Program (HITAP) for their inputs on the study; Ruobing Mei from the Laboratory of Data Discovery for Health (D4H) for her input on the study. 2 Publisher Copyright: {\textcopyright} 2023, The Author(s).",

year = "2023",

month = dec,

doi = "10.1038/s41746-023-00843-6",

language = "English",

volume = "6",

journal = "npj Digital Medicine",

issn = "2398-6352",

publisher = "Nature Publishing Group",

number = "1",

}

Lee, KY, Dabak, SV, Kong, VH, Park, M, Kwok, SLL, Silzle, M, Rachatan, C, Cook, A, Passanante, A, Pertwee, E, Wu, Z, Elkin, JA, Larson, HJ, Lau, EHY, Leung, K, Wu, JT & Lin, L 2023, 'Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore', npj Digital Medicine, vol. 6, no. 1, 96. https://doi.org/10.1038/s41746-023-00843-6

TY - JOUR

T1 - Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore

AU - Lee, Kristi Yoonsup

AU - Dabak, Saudamini Vishwanath

AU - Kong, Vivian Hanxiao

AU - Park, Minah

AU - Kwok, Shirley L.L.

AU - Silzle, Madison

AU - Rachatan, Chayapat

AU - Cook, Alex

AU - Passanante, Aly

AU - Pertwee, Ed

AU - Wu, Zhengdong

AU - Elkin, Javier A.

AU - Larson, Heidi J.

AU - Lau, Eric H.Y.

AU - Leung, Kathy

AU - Wu, Joseph T.

AU - Lin, Leesa

N1 - Funding Information: This work was supported by the Vaccine Confidence Fund (#VCF-020) and AIR@InnoHK administered by Innovation and Technology Commission. The Health Intervention and Technology Assessment Program (HITAP) was supported by a grant from the Access and Delivery Partnership (ADP), hosted by the United Nations Development Programme (UNDP) and funded by the Government of Japan. We thank Supatnuj Sorndamrih, Wattaporn Thanomsing, Nicha Krishnamra and Sindh R from the Thai Health Promotion Foundation (ThaiHealth) for their support on using the chatbot in Thailand; Dr. Yot Teerawattananon, Assoc. Prof. Wanrudee Isaranuwatchai, Dr. Pritaporn Kingkaew and Ms. Benjarin Santatiwongchai from the Health Intervention and Technology Assessment Program (HITAP) for their inputs on the study; Ruobing Mei from the Laboratory of Data Discovery for Health (D24H) for her input on the study. Funding Information: This work was supported by the Vaccine Confidence Fund (#VCF-020) and AIR@InnoHK administered by Innovation and Technology Commission. The Health Intervention and Technology Assessment Program (HITAP) was supported by a grant from the Access and Delivery Partnership (ADP), hosted by the United Nations Development Programme (UNDP) and funded by the Government of Japan. We thank Supatnuj Sorndamrih, Wattaporn Thanomsing, Nicha Krishnamra and Sindh R from the Thai Health Promotion Foundation (ThaiHealth) for their support on using the chatbot in Thailand; Dr. Yot Teerawattananon, Assoc. Prof. Wanrudee Isaranuwatchai, Dr. Pritaporn Kingkaew and Ms. Benjarin Santatiwongchai from the Health Intervention and Technology Assessment Program (HITAP) for their inputs on the study; Ruobing Mei from the Laboratory of Data Discovery for Health (D4H) for her input on the study. 2 Publisher Copyright: © 2023, The Author(s).

PY - 2023/12

Y1 - 2023/12

N2 - Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots’ significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

AB - Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots’ significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

UR - http://www.scopus.com/inward/record.url?scp=85160206847&partnerID=8YFLogxK

U2 - 10.1038/s41746-023-00843-6

DO - 10.1038/s41746-023-00843-6

M3 - Article

AN - SCOPUS:85160206847

SN - 2398-6352

VL - 6

JO - npj Digital Medicine

JF - npj Digital Medicine

IS - 1

M1 - 96

ER -

Effectiveness of chatbots on COVID vaccine confidence and acceptance in Thailand, Hong Kong, and Singapore

Abstract

Bibliographical note

UN SDGs

Access to Document

Fingerprint

Cite this