[Paper Review] Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objects

Ali Cheraghian, Shafin Rahman+2 more|arXiv (Cornell University)|Jul 15, 2019

Domain Adaptation and Few-Shot Learning43 references21 citations

TL;DR

This paper proposes a novel skewness-based loss to mitigate the hubness problem in zero-shot learning (ZSL) for 3D point cloud recognition, where poor-quality 3D features exacerbate label bias. By minimizing prediction skewness within each training batch, the method achieves state-of-the-art performance on ModelNet40, ModelNet10, McGill, and SHREC2015, establishing the first comprehensive evaluation of both ZSL and generalized ZSL in 3D vision.

ABSTRACT

The development of advanced 3D sensors has enabled many objects to be captured in the wild at a large scale, and a 3D object recognition system may therefore encounter many objects for which the system has received no training. Zero-Shot Learning (ZSL) approaches can assist such systems in recognizing previously unseen objects. Applying ZSL to 3D point cloud objects is an emerging topic in the area of 3D vision, however, a significant problem that ZSL often suffers from is the so-called hubness problem, which is when a model is biased to predict only a few particular labels for most of the test instances. We observe that this hubness problem is even more severe for 3D recognition than for 2D recognition. One reason for this is that in 2D one can use pre-trained networks trained on large datasets like ImageNet, which produces high-quality features. However, in the 3D case there are no such large-scale, labelled datasets available for pre-training which means that the extracted 3D features are of poorer quality which, in turn, exacerbates the hubness problem. In this paper, we therefore propose a loss to specifically address the hubness problem. Our proposed method is effective for both Zero-Shot and Generalized Zero-Shot Learning, and we perform extensive evaluations on the challenging datasets ModelNet40, ModelNet10, McGill and SHREC2015. A new state-of-the-art result for both zero-shot tasks in the 3D case is established.

Motivation & Objective

• To address the severe hubness problem in zero-shot learning (ZSL) for 3D point cloud recognition, which is exacerbated by low-quality 3D features due to lack of large-scale pre-training.
• To propose a new loss function that explicitly minimizes hubness by reducing prediction skewness in each training batch.
• To evaluate both standard ZSL and generalized ZSL (GZSL) for 3D point cloud classification, a setting previously unexplored in 3D vision.
• To demonstrate the effectiveness of the proposed method across multiple 3D benchmarks, including ModelNet40, ModelNet10, McGill, and SHREC2015.
• To show that projecting from features to semantics is more effective than the reverse, due to reduced hubness in the feature space.

Proposed method

• The method introduces a novel unsupervised loss, L_U, that computes the skewness of predicted class frequencies within each training batch to quantify hubness.
• The loss is minimized during training to reduce the tendency of the model to repeatedly predict a few high-frequency classes (hubs).
• Skewness is calculated as the normalized third central moment of the prediction distribution across seen classes in a batch, serving as a proxy for hubness.
• The method is applied during end-to-end training of a 3D point cloud classifier, with no need for additional supervision or pre-processing.
• The approach is compatible with both standard ZSL and generalized ZSL (GZSL), where both seen and unseen classes are present during inference.
• The method is evaluated with both GloVe and w2v word embeddings, showing that w2v yields better alignment with 3D point cloud semantics.

Experimental results

Research questions

• RQ1 How does the hubness problem manifest in 3D zero-shot learning, and why is it more severe than in 2D image-based ZSL?
• RQ2 Can a training-time loss that minimizes prediction skewness effectively reduce hubness in 3D point cloud recognition?
• RQ3 Is the proposed method effective in both standard ZSL and generalized ZSL (GZSL) settings for 3D data?
• RQ4 Which semantic embedding (GloVe vs. w2v) leads to better performance in 3D ZSL, and why?
• RQ5 Does projecting from features to semantics reduce hubness more effectively than the reverse direction?

Key findings

• The proposed skewness loss achieves a new state-of-the-art accuracy of 33.9% on ModelNet10, 12.5% on McGill, and 6.2% on SHREC2015 using GloVe embeddings.
• With w2v embeddings, the method achieves 28.7% accuracy on ModelNet10, 11.1% on McGill, and 4.2% on SHREC2015, outperforming prior methods.
• The method significantly reduces hubness, as evidenced by a drop in skewness values across both feature-to-semantic and semantic-to-feature projection settings.
• The method establishes the first comprehensive evaluation of generalized zero-shot learning (GZSL) on 3D point cloud datasets, including ModelNet40, ModelNet10, McGill, and SHREC2015.
• On the CUB 2D image dataset, the method achieves 59.4% accuracy, surpassing the previous SOTA of 58.3% by Zhang et al. (2017), demonstrating generalizability beyond 3D.
• w2v embeddings outperform GloVe in 3D ZSL, likely due to better alignment with the geometric semantics of point cloud data.