How Many Alternatives Can Be Ranked? A Comparison of the Paired Comparison and Ranking Methods

Minsu Ock, Nari Yi, Jeonghoon Ahn, Min Woo Jo

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

Objectives To determine the feasibility of converting ranking data into paired comparison (PC) data and suggest the number of alternatives that can be ranked by comparing a PC and a ranking method. Methods Using a total of 222 health states, a household survey was conducted in a sample of 300 individuals from the general population. Each respondent performed a PC 15 times and a ranking method 6 times (two attempts of ranking three, four, and five health states, respectively). The health states of the PC and the ranking method were constructed to overlap each other. We converted the ranked data into PC data and examined the consistency of the response rate. Applying probit regression, we obtained the predicted probability of each method. Pearson correlation coefficients were determined between the predicted probabilities of those methods. The mean absolute error was also assessed between the observed and the predicted values. Results The overall consistency of the response rate was 82.8%. The Pearson correlation coefficients were 0.789, 0.852, and 0.893 for ranking three, four, and five health states, respectively. The lowest mean absolute error was 0.082 (95% confidence interval [CI] 0.074–0.090) in ranking five health states, followed by 0.123 (95% CI 0.111–0.135) in ranking four health states and 0.126 (95% CI 0.113–0.138) in ranking three health states. Conclusions After empirically examining the consistency of the response rate between a PC and a ranking method, we suggest that using five alternatives in the ranking method may be superior to using three or four alternatives.

Original languageEnglish
Pages (from-to)655-660
Number of pages6
JournalValue in Health
Volume19
Issue number5
DOIs
StatePublished - 1 Jul 2016

Keywords

  • consistency
  • discrete choice experiments
  • paired comparison
  • ranking

Fingerprint

Dive into the research topics of 'How Many Alternatives Can Be Ranked? A Comparison of the Paired Comparison and Ranking Methods'. Together they form a unique fingerprint.

Cite this