Development and validation of practice-based multimedia assisted item: a mixed-method study

Background: The rapid development of information and technology and the spread of COVID-19 have made online learning and assessment more prevalent. In this context, the potential of multimedia-assisted items to provide an alternative method of learning and evaluation is significant. These items can better reflect authentic clinical situations and enhance higher cognitive skills. This study aims to develop and validate multimedia-assisted items that reflect clinical authenticity. Methods: A mixed-method design with qualitative and quantitative approaches was adopted. The study was conducted in two phases: development and validation. Thirty-six multimedia-assisted items were developed using videos, audio files, photos, and pictures. Preliminary testing ensured item comprehensibility and usability. In the validation phase, a mock test was administered to 136 nursing students, with 105 completing the survey. Quantitative analysis included calculating item difficulty, discrimination, and reliability and examining evidence based on relation with another variable through correlations with clinical reasoning scores and differences in test scores based on perceived academic achievement. Qualitative analysis involved thematic analysis of interviews with 10 participants to explore their experiences and validate the response process. Results: The average total score was 18.1 out of 36, with item difficulty averaging 0.51, item discrimination 0.31, and reliability (Cronbach’s alpha) 0.64. While no significant correlation was found between mock test scores and clinical reasoning scores, a significant difference was observed between high and low-perceived academic achievement groups. Participants expressed that the multimedia-assisted items were more relevant to nursing practice than paper-based items, integrating theoretical knowledge and clinical practicum experience, enhancing understanding, and stimulating active participation in clinical practicum. Conclusions: Multimedia-assisted items have been verified as an effective strategy for education and for assessing nursing students’ comprehensive competencies. MAIs are adequate for evaluating nursing students' competencies in theory and practice, suggesting that MAIs can enhance newly graduated nurses' competencies and improve online assessments in nursing education.