Deep self-correlation descriptor for dense cross-modal correspondence

Seungryong Kim, Dongbo Min, Stephen Lin, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

17 Scopus citations


We present a novel descriptor, called deep self-correlation (DSC), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. Motivated by local self-similarity (LSS), we formulate a novel descriptor by leveraging LSS in a deep architecture, leading to better discriminative power and greater robustness to non-rigid image deformations than state-of-the-art descriptors. The DSC first computes self-correlation surfaces over a local support window for randomly sampled patches, and then builds hierarchical self-correlation surfaces by performing an average pooling within a deep architecture. Finally, the feature responses on the self-correlation surfaces are encoded through a spatial pyramid pooling in a circular configuration. In contrast to convolutional neural networks (CNNs) based descriptors, the DSC is trainingfree, is robust to cross-modal imaging, and can be densely computed in an efficient manner that significantly reduces computational redundancy. The state-of-the-art performance of DSC on challenging cases of cross-modal image pairs is demonstrated through extensive experiments.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
PublisherSpringer Verlag
Number of pages17
ISBN (Print)9783319464831
StatePublished - 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: 8 Oct 201616 Oct 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9912 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference14th European Conference on Computer Vision, ECCV 2016

Bibliographical note

Publisher Copyright:
© Springer International Publishing AG 2016.


  • Cross-modal correspondence
  • Deep architecture
  • Local self-similarity
  • Non-rigid deformation
  • Selfcorrelation


Dive into the research topics of 'Deep self-correlation descriptor for dense cross-modal correspondence'. Together they form a unique fingerprint.

Cite this