[Paper Review] Robust Line Segments Matching via Graph Convolution Networks

Quanmeng Ma, Guang Jiang+1 more|arXiv (Cornell University)|Apr 10, 2020

Advanced Image and Video Retrieval Techniques28 references9 citations

TL;DR

This paper proposes an end-to-end graph convolutional network (GCN)-based method for robust line segment matching in low-texture and challenging scenes, jointly learning local line descriptors and matching via intra- and cross-graph convolutions with optimal transport. The approach achieves 70.47% recall—25% higher than the prior state-of-the-art—by leveraging top-k pooling for robust graph architecture learning and GLpooling for endpoint-invariant feature extraction.

ABSTRACT

Line matching plays an essential role in structure from motion (SFM) and simultaneous localization and mapping (SLAM), especially in low-textured and repetitive scenes. In this paper, we present a new method of using a graph convolution network to match line segments in a pair of images, and we design a graph-based strategy of matching line segments with relaxing to an optimal transport problem. In contrast to hand-crafted line matching algorithms, our approach learns local line segment descriptor and the matching simultaneously through end-to-end training. The results show our method outperforms the state-of-the-art techniques, and especially, the recall is improved from 45.28% to 70.47% under a similar presicion. The code of our work is available at https://github.com/mameng1/GraphLineMatching.

Motivation & Objective

• Address the challenge of matching line segments in low-texture, repetitive, and view-invariant indoor scenes where traditional point-based methods fail.
• Overcome limitations of hand-crafted descriptors and geometric constraints that are sensitive to endpoint inaccuracies and viewpoint changes.
• Develop a unified deep learning framework that jointly learns line segment descriptors and correspondence matching in an end-to-end manner.
• Improve robustness to image transformations such as rotation, scaling, blurring, and illumination changes through invariant feature learning and graph-based context aggregation.

Proposed method

• Introduces a line feature learning module that extracts features from a rectangular region centered on each line segment, reducing sensitivity to endpoint inaccuracies.
• Employs a graph convolution network with three components: graph architecture learning via top-k pooling, intra-graph convolution for local context, and cross-graph convolution for inter-image feature alignment.
• Uses a learnable top-k pooling strategy to build robust, sparse graph connections that avoid invalid relationships, especially in datasets with unmatched lines.
• Applies optimal transport to compute an assignment matrix between line sets, enabling differentiable, soft matching with differentiable filtering of non-matches.
• Incorporates a feature learning loss that enhances descriptor compactness for matched lines and discrepancy for unmatched lines, improving discrimination.
• Uses GLpooling to aggregate features from points along line segments, producing more stable and invariant descriptors under small endpoint shifts.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework jointly learn local line segment descriptors and matching decisions more effectively than hand-crafted or two-stage methods?
• RQ2 How does graph-based context aggregation improve matching robustness in low-texture and view-invariant scenes?
• RQ3 To what extent does top-k pooling-based graph architecture learning enhance performance on datasets with unmatched or fragmented lines?
• RQ4 How does the proposed feature learning loss and GLpooling contribute to invariance against endpoint inaccuracies and image transformations?
• RQ5 Can the method achieve significantly higher recall than state-of-the-art techniques while maintaining high precision under challenging conditions?

Key findings

• The proposed method achieves 70.47% recall, a 25.19% relative improvement over the prior state-of-the-art method LJL (45.29% recall), under similar precision (86.12% vs. 85.46%).
• The method maintains high precision (86.12%) and achieves an F-measure of 77.51%, outperforming all baselines including LJL, LPI, and MSLD.
• Ablation studies show that removing top-k graph learning reduces recall by 2.63% (69.46% → 66.83%), confirming its importance in handling unmatched lines.
• GLpooling improves recall by 2.63% compared to using raw line-point features, demonstrating its effectiveness in reducing sensitivity to endpoint errors.
• The method shows superior robustness to rotation, scaling, and blurring—especially in recall—outperforming LJL across all transformation types.
• Qualitative results confirm that the method generates more correct matches (blue lines) and fewer incorrect matches (red lines), particularly in low-texture and blurred scenes.