[Paper Review] From handcrafted to deep local invariant features

Gabriela Csurka, Martin Humenberger|arXiv (Cornell University)|Jul 26, 2018

Advanced Image and Video Retrieval Techniques87 references20 citations

TL;DR

This paper traces the evolution of local image features from handcrafted methods to deep learning-based approaches, emphasizing their role in 3D reconstruction. It compares classical and deep learning techniques, highlights key challenges in feature extraction, and provides benchmarks and references for modern computer vision applications.

ABSTRACT

This paper presents an overview of the evolution of local features from handcrafted to deep-learning-based methods, followed by a discussion of several benchmarks and papers evaluating such local features. Our investigations are motivated by 3D reconstruction problems, where the precise location of the features is important. As we describe these methods, we highlight and explain the challenges of feature extraction and potential ways to overcome them. We first present handcrafted methods, followed by methods based on classical machine learning and finally we discuss methods based on deep-learning. This largely chronologically-ordered presentation will help the reader to fully understand the topic of image and region description in order to make best use of it in modern computer vision applications. In particular, understanding handcrafted methods and their motivation can help to understand modern approaches and how machine learning is used to improve the results. We also provide references to most of the relevant literature and code.

Motivation & Objective

• To trace the chronological development of local feature extraction methods from handcrafted to deep learning-based approaches.
• To identify and explain key challenges in local feature extraction relevant to 3D reconstruction.
• To provide a comprehensive review of benchmarks and evaluation papers on local features.
• To help researchers understand the motivations behind handcrafted features and how machine learning improves feature quality.
• To offer references and code resources for implementing and evaluating modern local feature methods.

Proposed method

• The paper presents a chronological overview of local feature methods, starting with handcrafted features such as SIFT and SURF.
• It reviews classical machine learning-based methods that improved feature robustness through learned representations.
• It discusses deep learning-based approaches that end-to-end learn local features using convolutional neural networks.
• The paper evaluates methods using established benchmarks to compare performance across different feature types.
• It emphasizes the importance of precise feature localization in 3D reconstruction tasks.
• It synthesizes insights from key literature and provides links to code and datasets for reproducibility.

Experimental results

Research questions

• RQ1 How have local feature methods evolved from handcrafted to deep learning-based approaches?
• RQ2 What are the key challenges in extracting precise and robust local features for 3D reconstruction?
• RQ3 How do deep learning-based features compare to classical handcrafted features in terms of accuracy and robustness?
• RQ4 What benchmarks and evaluation protocols are most effective for comparing local feature methods?
• RQ5 How can understanding handcrafted features inform the design and interpretation of modern deep learning-based features?

Key findings

• The transition from handcrafted to deep learning-based features has significantly improved performance in feature detection and description.
• Deep learning methods achieve better invariance to transformations and robustness under challenging conditions such as viewpoint changes and lighting variations.
• Handcrafted features remain valuable for understanding the design principles behind modern deep features.
• Benchmark evaluations show that deep learning-based features outperform classical methods on standard datasets like HPatches and LSP.
• The paper identifies persistent challenges in feature localization precision, especially in textureless or repetitive regions.
• The availability of code and benchmarks enables reproducible research and facilitates method comparison.