Developing a scoring guide for the appraisal of guidelines for research and evaluation II instrument in Korea: A modified delphi consensus process

Korea has a relatively short history in the development and use of clinical practice guidelines (CPGs). Additionally, it has been difficult to employ the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument due to the lack of consensus and the presence of differences in Korean medical settings and in the Korean socio-cultural environment. An AGREE II scoring guide was therefore developed to reduce differences among evaluators using the same tool. In consideration of the importance of using a quantitative measure of satisfaction with the elements described in the AGREE II manual, a final draft was developed through a Delphi consensus process. Ninety-two draft scoring guides for anchor points 1, 3, 5, and 7 (full score) in 23 items were developed. Consensus was defined as agreement among at least 70% of the raters. Agreement on 88 draft scoring guidelines was reached in the first Delphi round, and agreement for the remaining four was achieved in the second round. The development of an AGREE II scoring guide in this study is expected to contribute to improving the CPG environment.