[Paper Review] Dynamic Graph Representation for Partially Occluded Biometrics

Min Ren, Yunlong Wang+2 more|arXiv (Cornell University)|Dec 1, 2019

Biometric Identification and Security40 references4 citations

TL;DR

This paper proposes Dynamic Graph Representation (DGR), a novel deep learning framework that combines CNNs and graphical models to handle partial occlusion in biometrics. By constructing dynamic feature graphs from CNN activations and adaptively removing nodes corresponding to occluded regions, DGR improves recognition accuracy and provides interpretable, reasoning-based decisions, outperforming CNNs and masking strategies on iris and face recognition benchmarks with significant margins.

ABSTRACT

The generalization ability of Convolutional neural networks (CNNs) for biometrics drops greatly due to the adverse effects of various occlusions. To this end, we propose a novel unified framework integrated the merits of both CNNs and graphical models to learn dynamic graph representations for occlusion problems in biometrics, called Dynamic Graph Representation (DGR). Convolutional features onto certain regions are re-crafted by a graph generator to establish the connections among the spatial parts of biometrics and build Feature Graphs based on these node representations. Each node of Feature Graphs corresponds to a specific part of the input image and the edges express the spatial relationships between parts. By analyzing the similarities between the nodes, the framework is able to adaptively remove the nodes representing the occluded parts. During dynamic graph matching, we propose a novel strategy to measure the distances of both nodes and adjacent matrixes. In this way, the proposed method is more convincing than CNNs-based methods because the dynamic graph method implies a more illustrative and reasonable inference of the biometrics decision. Experiments conducted on iris and face demonstrate the superiority of the proposed framework, which boosts the accuracy of occluded biometrics recognition by a large margin comparing with baseline methods.The code is avaliable at https://github.com/RenMin1991/Dyamic\_Graph\_Representation

Motivation & Objective

• To address the severe drop in generalization performance of CNNs in biometric recognition under partial occlusion.
• To overcome the limitations of masking strategies, which are error-prone and harmful when applied post-feature extraction.
• To develop a unified framework that integrates CNNs and graphical models for robust occlusion handling.
• To provide interpretable, reasoning-based decisions by visualizing node similarities and structural matches in feature graphs.
• To improve intra-class similarity and generalization in unconstrained biometric scenarios.

Proposed method

• A graph generator extracts spatial regions from CNN feature maps and establishes connections between them to form Feature Graphs, where nodes represent image parts and edges encode spatial relationships.
• A novel Squeeze-and-Excitation Graph Attention Network (SE-GAT) processes the Feature Graphs to refine node representations using attention mechanisms.
• During matching, the framework dynamically removes nodes corresponding to occluded regions based on similarity analysis, enabling adaptive and robust comparison.
• A dual-distance matching strategy computes similarity between node features and adjacency matrices, enhancing discriminative power.
• The framework is end-to-end trainable and integrates CNN-based feature extraction with differentiable graph operations.
• The method avoids pre-masking by learning to ignore occluded regions through graph structure adaptation rather than feature suppression.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework combining CNNs and graphical models improve recognition accuracy under partial occlusion in biometrics?
• RQ2 Does dynamic graph construction with adaptive node removal outperform fixed masking strategies in occluded biometric recognition?
• RQ3 Can graphical representations provide more interpretable and reasonable decision reasoning than standard CNNs?
• RQ4 How does the proposed method affect intra-class similarity and generalization in unconstrained biometric settings?
• RQ5 What is the optimal number of nodes in the feature graph for balancing accuracy and computational cost?

Key findings

• On the Occluded-LFW dataset with 30% occlusion, the proposed method achieved 71.8% TPR at FRR=0.1%, significantly outperforming LCNN-9 (22.47%).
• With 50% occlusion, the method achieved 25.19% TPR at FRR=0.1%, compared to 8.47% for LCNN-9, demonstrating robustness to severe occlusion.
• On the ND CrossSensor Iris 2013 Dataset-LG4000, the method achieved an EER of 0.62% with 32 nodes, outperforming the baseline and masking strategies.
• Experiments showed that masking occluded regions before feature extraction harms performance, indicating that masking is detrimental in end-to-end deep learning frameworks.
• Feature graph visualization confirmed that the model focuses on the most similar regions (e.g., around the eyes), providing interpretable reasoning for matching decisions.
• Parameter analysis revealed that 32 nodes offered the best trade-off between accuracy and computational cost, with EER of 0.62%.