[Paper Review] Hierarchical Adaptive Structural SVM for Domain Adaptation

Jiaolong Xu, Sebastian Ramos+2 more|arXiv (Cornell University)|Aug 22, 2014

Domain Adaptation and Few-Shot Learning32 references10 citations

TL;DR

This paper proposes Hierarchical Adaptive Structural SVM (HA-SSVM), a novel domain adaptation method that leverages a hierarchical structure of multiple target domains or sub-domains to improve classification accuracy. By jointly adapting a source-domain classifier through a tree-structured hierarchy of target domains using adaptive structural SVM, HA-SSVM outperforms single-layer adaptation strategies in pedestrian detection and multi-category object recognition, even when sub-domains are discovered automatically.

ABSTRACT

A key topic in classification is the accuracy loss produced when the data distribution in the training (source) domain differs from that in the testing (target) domain. This is being recognized as a very relevant problem for many computer vision tasks such as image classification, object detection, and object category recognition. In this paper, we present a novel domain adaptation method that leverages multiple target domains (or sub-domains) in a hierarchical adaptation tree. The core idea is to exploit the commonalities and differences of the jointly considered target domains. Given the relevance of structural SVM (SSVM) classifiers, we apply our idea to the adaptive SSVM (A-SSVM), which only requires the target domain samples together with the existing source-domain classifier for performing the desired adaptation. Altogether, we term our proposal as hierarchical A-SSVM (HA-SSVM). As proof of concept we use HA-SSVM for pedestrian detection and object category recognition. In the former we apply HA-SSVM to the deformable part-based model (DPM) while in the latter HA-SSVM is applied to multi-category classifiers. In both cases, we show how HA-SSVM is effective in increasing the detection/recognition accuracy with respect to adaptation strategies that ignore the structure of the target data. Since, the sub-domains of the target data are not always known a priori, we shown how HA-SSVM can incorporate sub-domain structure discovery for object category recognition.

Motivation & Objective

• Address the challenge of domain shift in computer vision where training (source) and test (target) data distributions differ.
• Overcome limitations of single-layer domain adaptation methods that treat each target domain in isolation or pool all targets into one.
• Develop a method that exploits both commonalities across related target sub-domains and their unique differences through a hierarchical structure.
• Enable domain adaptation without requiring access to source-domain data, relying only on labeled target samples and the pre-trained source classifier.
• Demonstrate effectiveness in both known and discovered target domain structures, particularly in object recognition with latent domain discovery.

Proposed method

• Construct a hierarchical adaptation tree where target domains are organized into parent-child relationships based on their similarity.
• Apply adaptive structural SVM (A-SSVM) to jointly optimize model parameters across all nodes in the hierarchy, from root to leaves.
• Use a single objective function that enables progressive adaptation: intermediate nodes capture shared features across sub-domains, while leaf nodes model domain-specific variations.
• Enable adaptation with only target-domain labeled data and the pre-trained source classifier, avoiding re-access to source data.
• Incorporate sub-domain discovery via clustering (e.g., Reshape, LatDD) when domain structure is unknown a priori, using only 3 labeled examples per category per sub-domain.
• Train multi-category classifiers jointly across all hierarchy levels, allowing error compensation from imperfect domain assignments.

Experimental results

Research questions

• RQ1 Can hierarchical domain adaptation improve performance over single-layer adaptation in scenarios with multiple correlated target domains?
• RQ2 How does exploiting shared structure across sub-domains affect detection and recognition accuracy compared to pooling all target data?
• RQ3 To what extent can HA-SSVM maintain high performance when target sub-domains are not known a priori but must be discovered?
• RQ4 Does joint optimization across the hierarchy mitigate errors from inaccurate sub-domain discovery?
• RQ5 How does HA-SSVM compare to state-of-the-art domain adaptation methods in pedestrian detection and multi-category object recognition tasks?

Key findings

• HA-SSVM achieves a pedestrian detection mAP of 59.1% on the PASCAL VOC 2007 test set using a three-layer hierarchy, outperforming single-layer pooling strategies.
• In multi-category object recognition, HA-SSVM with discovered sub-domains (Reshape-Pr) achieves 49.1% average accuracy, comparable to the 49.0% accuracy when domains are given a priori.
• The method outperforms both single-layer adaptation (independent adaptation per domain) and pooling-based strategies, even with limited labeled data per sub-domain.
• Reshape-Pr, which uses ground-truth-based domain discovery, yields higher accuracy than LatDD-Pr, but HA-SSVM’s joint training partially compensates for domain misclassification errors.
• The three-layer HA-SSVM configuration achieves results on par with the best two-layer setups, demonstrating the scalability and robustness of the hierarchical approach.
• When sub-domains are discovered via Reshape-Pr, HA-SSVM maintains strong performance (49.1% accuracy), showing that the method is effective even under real-world conditions with unknown domain structures.