[Paper Review] Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation

Liang Chen, Wenguan Wang+2 more|arXiv (Cornell University)|Aug 24, 2023

Natural Language Processing TechniquesComputer Science3 citations

TL;DR

This paper proposes LOGICDIAG, a neural-symbolic semi-supervised learning framework that uses first-order logic rules to detect and resolve conflicts in pseudo-labeled predictions, improving semantic segmentation accuracy by leveraging symbolic reasoning to correct erroneous labels. The method achieves state-of-the-art performance with mIoU gains of 1.21%–4.15% across three benchmarks, especially in low-label settings.

ABSTRACT

Recent advances in semi-supervised semantic segmentation have been heavily reliant on pseudo labeling to compensate for limited labeled data, disregarding the valuable relational knowledge among semantic concepts. To bridge this gap, we devise LogicDiag, a brand new neural-logic semi-supervised learning framework. Our key insight is that conflicts within pseudo labels, identified through symbolic knowledge, can serve as strong yet commonly ignored learning signals. LogicDiag resolves such conflicts via reasoning with logic-induced diagnoses, enabling the recovery of (potentially) erroneous pseudo labels, ultimately alleviating the notorious error accumulation problem. We showcase the practical application of LogicDiag in the data-hungry segmentation scenario, where we formalize the structured abstraction of semantic concepts as a set of logic rules. Extensive experiments on three standard semi-supervised semantic segmentation benchmarks demonstrate the effectiveness and generality of LogicDiag. Moreover, LogicDiag highlights the promising opportunities arising from the systematic integration of symbolic reasoning into the prevalent statistical, neural learning approaches.

Motivation & Objective

• To address the error accumulation problem in semi-supervised semantic segmentation caused by unreliable pseudo labels.
• To integrate symbolic knowledge—expressed as first-order logic rules—into neural network training to improve model robustness and interpretability.
• To develop a diagnostic reasoning mechanism that identifies and resolves conflicts in pseudo labels using logic-based diagnoses.
• To demonstrate that structured symbolic knowledge, such as composition, decomposition, and exclusion rules, significantly enhances model generalization in data-scarce scenarios.

Proposed method

• The framework employs a teacher-student training paradigm with confidence thresholding for initial pseudo labeling.
• It formulates semantic relationships (e.g., Cat → Animal, Animal ⊥ Vehicle) as first-order logic rules to encode relational knowledge.
• It detects prediction conflicts by checking consistency between model outputs and symbolic logic rules.
• It computes diagnosis likelihoods using fuzzy logic that combine predictive confidence and conflict severity to guide correction.
• It resolves conflicts via Monte Carlo sampling from a posterior distribution over possible diagnoses, enabling robust correction of erroneous predictions.
• The method integrates seamlessly into existing SSL frameworks with minimal modification, only requiring a hierarchical classification head and logic rule injection.

Experimental results

Research questions

• RQ1 Can symbolic logic rules be effectively used to detect and correct errors in pseudo-labeled segmentation outputs?
• RQ2 How does integrating logic-based conflict diagnosis improve performance in low-data semi-supervised learning?
• RQ3 What is the relative contribution of different types of semantic rules—composition, decomposition, and exclusion—to model performance?
• RQ4 How does the fuzzy logic-based diagnosis weighting strategy compare to heuristic or uniform resolution methods?
• RQ5 To what extent can logic-induced reasoning reduce confirmation bias and error accumulation in iterative self-training?

Key findings

• LOGICDIAG achieves a 4.15% mIoU improvement over baseline methods on the PASCAL VOC 2012 dataset with only 1/16 of the training labels.
• On Cityscapes, the method achieves a 1.65% average mIoU gain across all label partitions, demonstrating strong generalization in complex urban scenes.
• On COCO, LOGICDIAG improves performance by 2.94% mIoU on average over the state-of-the-art MKD method, highlighting its effectiveness in large-scale, hierarchical datasets.
• Ablation studies confirm that all three logic rules—Composition, Decomposition, and Exclusion—contribute independently to performance gains, with the full rule set yielding the best results.
• The 'Sampling' resolution strategy, which uses posterior estimation via Monte Carlo sampling, outperforms 'Uniform', 'Predictive', and 'Greedy' strategies, indicating its superiority in escaping spurious corrections.
• The training overhead is minimal, adding only 4.7% inference time, and no computational cost is incurred at test time due to ancestral class pruning.