Rediscovery rate estimation for assessing the validation of significant findings in high-throughput studies

Andrea Ganna, Donghwan Lee, Erik Ingelsson, Yudi Pawitan

Research output: Contribution to journalArticlepeer-review

25 Scopus citations

Abstract

It is common and advised practice in biomedical research to validate experimental or observational findings in a population different from the one where the findings were initially assessed. This practice increases the generalizability of the results and decreases the likelihood of reporting false-positive findings. Validation becomes critical when dealing with high-throughput experiments, where the large number of tests increases the chance to observe false-positive results. In this article, we review common approaches to determine statistical thresholds for validation and describe the factors influencing the proportion of significant findings from a 'training' sample that are replicated in a 'validation' sample.We refer to this proportion as rediscovery rate (RDR). In high-throughput studies, the RDR is a function of false-positive rate and power in both the training and validation samples.We illustrate the application of the RDR using simulated data and real data examples from metabolomics experiments.We further describe an online tool to calculate the RDR using t-statistics.We foresee two main applications. First, if the validation study has not yet been collected, the RDR can be used to decide the optimal combination between the proportion of findings taken to validation and the size of the validation study. Secondly, if a validation study has already been done, the RDR estimated using the training data can be compared with the observed RDR from the validation data; hence, the success of the validation study can be assessed.

Original languageEnglish
Pages (from-to)563-575
Number of pages13
JournalBriefings in Bioinformatics
Volume16
Issue number4
DOIs
StatePublished - 10 Jul 2014

Keywords

  • False discovery rate
  • Metabolomics
  • Multiple testing
  • Rediscovery rate
  • Statistical validation

Fingerprint

Dive into the research topics of 'Rediscovery rate estimation for assessing the validation of significant findings in high-throughput studies'. Together they form a unique fingerprint.

Cite this