[Paper Review] Efficient Mirror Detection via Multi-level Heterogeneous Learning

Ruozhen He, Jiaying Lin+1 more|arXiv (Cornell University)|Nov 28, 2022

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes HetNet, a highly efficient mirror detection network that uses multi-level heterogeneous learning to improve accuracy and speed by applying distinct modules—multi-orientation intensity-based contrasted (MIC) for low-level features and reflection semantic logical (RSL) for high-level features—resulting in 664% faster inference and 8.9% average improvement in MAE over state-of-the-art methods.

ABSTRACT

We present HetNet (Multi-level extbf{Het}erogeneous extbf{Net}work), a highly efficient mirror detection network. Current mirror detection methods focus more on performance than efficiency, limiting the real-time applications (such as drones). Their lack of efficiency is aroused by the common design of adopting homogeneous modules at different levels, which ignores the difference between different levels of features. In contrast, HetNet detects potential mirror regions initially through low-level understandings ( extit{e.g.}, intensity contrasts) and then combines with high-level understandings (contextual discontinuity for instance) to finalize the predictions. To perform accurate yet efficient mirror detection, HetNet follows an effective architecture that obtains specific information at different stages to detect mirrors. We further propose a multi-orientation intensity-based contrasted module (MIC) and a reflection semantic logical module (RSL), equipped on HetNet, to predict potential mirror regions by low-level understandings and analyze semantic logic in scenarios by high-level understandings, respectively. Compared to the state-of-the-art method, HetNet runs 664$\%$ faster and draws an average performance gain of 8.9$\%$ on MAE, 3.1$\%$ on IoU, and 2.0$\%$ on F-measure on two mirror detection benchmarks.

Motivation & Objective

• Address the inefficiency of existing mirror detection models that use homogeneous modules across feature levels, limiting real-time deployment.
• Overcome the performance-efficiency trade-off in mirror detection by exploiting the distinct characteristics of low-level and high-level features.
• Develop a model that mimics human visual processing by combining low-level intensity contrasts and high-level semantic reasoning for robust mirror localization.
• Reduce computational cost and eliminate reliance on post-processing (e.g., CRF) to enable real-time applications in drones and robotics.
• Achieve state-of-the-art performance with minimal FLOPs and high inference speed while maintaining high accuracy on standard benchmarks.

Proposed method

• Design a heterogeneous network architecture (HetNet) that splits backbone features into low-level and high-level representations for specialized processing.
• Introduce the multi-orientation intensity-based contrasted (MIC) module to detect mirror candidates using intensity contrasts in two orthogonal orientations (0° and 90°), minimizing information loss.
• Implement the reflection semantic logical (RSL) module to analyze contextual and semantic discontinuities (e.g., content similarity, reflection symmetry) at high-level features.
• Decouple low-level and high-level feature learning by applying distinct modules tailored to their spatial and semantic characteristics.
• Use a backbone network efficiently without redundant computation, preserving low-level details while extracting high-level semantics.
• Apply a rotation strategy with 90° increments to avoid tensor distortion and ensure balanced, comprehensive low-level feature extraction.

Experimental results

Research questions

• RQ1 Can a heterogeneous network design that treats low-level and high-level features with different modules improve both efficiency and accuracy in mirror detection?
• RQ2 How does multi-orientation intensity contrast analysis enhance low-level mirror candidate detection compared to single-direction or non-orthogonal methods?
• RQ3 To what extent can high-level semantic reasoning (e.g., contextual discontinuity, content similarity) improve final mirror mask prediction?
• RQ4 Can the proposed method achieve state-of-the-art performance with significantly reduced FLOPs and faster inference speed than existing methods?
• RQ5 What are the limitations of the model in detecting mirrors with complex distortions or non-regular reflection properties?

Key findings

• HetNet achieves a 664% speedup in inference (FPS) compared to the SOTA method PMD, with 49.23 FPS on the MSD dataset.
• The model reduces FLOPs by 72.73% compared to PMD, demonstrating substantial computational efficiency.
• HetNet improves MAE by 8.9% on average across two benchmarks, IoU by 3.1%, and F-measure by 2.0% over the previous SOTA method.
• Ablation studies confirm that both MIC and RSL modules are essential, with MIC contributing significantly to low-level detection and RSL to high-level reasoning.
• The multi-orientation strategy in MIC outperforms single-orientation or non-orthogonal configurations, with ICFE*4 and ICFE*3 showing worse performance due to redundancy or imbalance.
• The model fails on distorting mirrors with complex reflection properties, indicating a limitation in handling non-regular or highly distorted reflections.