[Paper Review] Dual-Resolution Correspondence Networks

Xinghui Li, Kai Han+2 more|arXiv (Cornell University)|Jun 16, 2020

Advanced Neural Network Applications51 references76 citations

TL;DR

DualRC-Net establishes dense pixel-wise correspondences in a coarse-to-fine manner using dual-resolution feature maps, achieving state-of-the-art results on HPatches, InLoc, and Aachen Day-Night while reducing memory and computation.

ABSTRACT

We tackle the problem of establishing dense pixel-wise correspondences between a pair of images. In this work, we introduce Dual-Resolution Correspondence Networks (DualRC-Net), to obtain pixel-wise correspondences in a coarse-to-fine manner. DualRC-Net extracts both coarse- and fine- resolution feature maps. The coarse maps are used to produce a full but coarse 4D correlation tensor, which is then refined by a learnable neighbourhood consensus module. The fine-resolution feature maps are used to obtain the final dense correspondences guided by the refined coarse 4D correlation tensor. The selected coarse-resolution matching scores allow the fine-resolution features to focus only on a limited number of possible matches with high confidence. In this way, DualRC-Net dramatically increases matching reliability and localisation accuracy, while avoiding to apply the expensive 4D convolution kernels on fine-resolution feature maps. We comprehensively evaluate our method on large-scale public benchmarks including HPatches, InLoc, and Aachen Day-Night. It achieves the state-of-the-art results on all of them.

Motivation & Objective

• Motivate robust dense matching across large viewpoint and illumination changes.
• Develop a scalable architecture that avoids full-resolution 4D tensors
• Leverage coarse-to-fine guidance to focus fine-resolution matching
• Improve localisation accuracy without prohibitive memory costs
• Demonstrate state-of-the-art performance on multiple public benchmarks (HPatches, InLoc, Aachen Day-Night)

Proposed method

• Extract dual-resolution feature maps with an FPN-like backbone to obtain coarse and fine features.
• Form a full 4D correlation tensor from coarse features and refine it with a neighbourhood consensus module.
• Guide fine-resolution matching using the refined coarse correlation tensor to select high-confidence local regions.
• Compute dense matches by combining fine-resolution scores with masked, upsampled coarse scores.
• Train end-to-end using sparse keypoint annotations with a combination of Frobenius-norm loss and an orthogonal regularizer.
• Speed up dense matching by querying only regions around high-coherence coarse matches rather than the full fine-resolution space.

Experimental results

Research questions

• RQ1 Can dual-resolution features enable reliable dense correspondences at high image resolutions?
• RQ2 Does coarse-to-fine guidance reduce memory and computation while preserving or improving localisation accuracy?
• RQ3 How does DualRC-Net perform on challenging benchmarks with occlusion, illumination, and viewpoint changes?
• RQ4 Does the method achieve state-of-the-art results across indoor and outdoor relocalization tasks?

Key findings

• Achieves state-of-the-art results on HPatches, InLoc, and Aachen Day-Night benchmarks.
• Utilizes coarse-resolution 4D correlation refined by neighbourhood consensus to guide fine-resolution matching, improving reliability and localisation.
• Reduces memory footprint and computation by restricting fine-resolution matching to regions flagged by coarse matches.
• Outperforms NCNet and Sparse-NCNet, particularly under severe illumination and viewpoint changes on HPatches.
• Demonstrates strong generalisation across diverse datasets including indoor and outdoor scenes.