Dynamics of relevance judgment during physicians’ online information search process for patient treatment

Soon Kim, Eun Kyung Chung, Nahyun Kwon

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Physicians rely on online sources of information more so than ever, to find the information they need when treating patients, but little is known about determining the relevance criteria when searching for online sources. Relevance judgment criteria can change with a user’s background knowledge on a search topic. Physicians are a subset of users who have received insufficient attention in this relevance criteria research topic. Due to the dynamic and multidimensional nature of relevance judgments, it is important to understand how relevance judgment criteria change in the course of information search. This study aims to determine the dynamics in relevance judgment criteria in physicians’ information search process. We observed ten internal medicine physicians while they searched the online clinical information resource to resolve a clinical question concerning patient treatment. They rated on a checklist to assess relevance judgment criteria at each of the three stages in the search process, namely, problem recognition (before searching), system interaction (searching for online resources), and document interaction (selecting the final documents after searching). We conducted pre-interview and post-interview to probe physicians search behaviours and reasons for the assessments. Topical relevance, accuracy, and credibility were found to be the most important criteria for all three search stages; however, ratings of some relevance judgment criteria, such as title, obtainability, personal preference, and understandability, changed markedly across the search process. The criteria regarding the information content tended to be more important during the problem recognition stage, while the criteria in the information format became more important as the search progressed. Physicians preferred recently published review articles in highly reliable journals with well-written article outlines, which allowed them to have quick and clear understanding of the contents. By identifying specific relevance criteria prominent at each search stage, our study revealed that physicians employ a unique set of criteria when judging the relevance. Given the fact that physicians are relying on online resources to find answers to patient care, this study can help in developing the effective clinical information systems for physicians and integrating into the Electronic Health Record (EHR) systems.

Original languageEnglish
Pages (from-to)17-37
Number of pages21
JournalMalaysian Journal of Library and Information Science
Issue number1
StatePublished - 2021

Bibliographical note

Funding Information:
This study draws on some parts of Soon Kim’s Ph.D. thesis: An investigation of physicians’ online information search process and the dynamics of relevance judgment. Seoul: Department of Library and Information Science, Ewha Womans University; 2016. No grant from any public, commercial, or non-profit funding agency was offered for the conduct of this research.

Publisher Copyright:
© 2021


  • Dynamics of relevance judgment
  • Information search process
  • Information seeking behaviour
  • Online clinical information systems
  • Online information search


Dive into the research topics of 'Dynamics of relevance judgment during physicians’ online information search process for patient treatment'. Together they form a unique fingerprint.

Cite this