Smooth Ride: Low-Pass Filtering of Manual Segmentations Improves Consensus

In this paper, we investigate slice-wise manual segmentation of knee anatomy. Due to high inter-rater variability between annotators, often a high number of raters is required to obtain a reliable ground truth consensus. We conducted an extensive study in which cartilage surface was segmented manually by six annotators on three scans of the knee. The slice-wise annotation results in high-frequency artifact that can be reduced by averaging over the segmentations of the annotators. A similar effect can also be obtained by smoothing the surface using lowpass filtering. In our results, we demonstrate that such filtering increases the consistency of the annotation of all raters. Furthermore, due to the smoothness of the cartilage surface, strong filtering produces surfaces that show differences to the ground truth that are in the same order of magnitude as the inter-rater variation. The remaining root mean squared error lies in the range of 0.11 to 0.14 mm. These findings show that appropriate pre-processing techniques result in segmentations close to the consensus of multiple raters, suggesting that in the future fewer annotators are required to achieve a reliable segmentation.