[Paper Review] A bag-to-class divergence approach to multiple-instance learning

Kajsa Møllersen, Jon Yngve Hardeberg+1 more|arXiv (Cornell University)|Mar 7, 2018

Image Retrieval and Classification Techniques31 references3 citations

TL;DR

This paper proposes a bag-to-class divergence approach for multi-instance learning that models bags as probability distributions, enabling the use of statistical divergences like class-conditional Kullback-Leibler (cKL) information to improve classification under data sparsity. By leveraging class-conditional distribution modeling, the method reduces computational cost and enhances performance on sparse MI datasets compared to bag-to-bag divergences.

ABSTRACT

In multi-instance (MI) learning, each object (bag) consists of multiple feature vectors (instances), and is most commonly regarded as a set of points in a multidimensional space. A different viewpoint is that the instances are realisations of random vectors with corresponding probability distribution, and that a bag is the distribution, not the realisations. In MI classification, each bag in the training set has a class label, but the instances are unlabelled. By introducing the probability distribution space to bag-level classification problems, dissimilarities between probability distributions (divergences) can be applied. The bag-to-bag Kullback-Leibler information is asymptotically the best classifier, but the typical sparseness of MI training sets is an obstacle. We introduce bag-to-class divergence to MI learning, emphasising the hierarchical nature of the random vectors that makes bags from the same class different. We propose two properties for bag-to-class divergences, and an additional property for sparse training sets.

Motivation & Objective

• Address the challenge of data sparsity in multi-instance learning, where bags often contain few instances.
• Overcome the high computational cost of bag-to-bag divergences like Kullback-Leibler information.
• Develop a statistically principled framework for multi-instance classification by modeling bags as probability distributions.
• Introduce a new class of dissimilarity measures—bag-to-class divergences—that incorporate class-conditional information.
• Provide a theoretical foundation for evaluating MI methods using properties of divergences independent of data.

Proposed method

• Model each bag as a probability distribution over instances, shifting focus from instance-level to distribution-level representation.
• Propose bag-to-class divergence using class-conditional Kullback-Leibler (cKL) information as a measure of dissimilarity between a bag's distribution and class-conditional distributions.
• Apply importance sampling to approximate the divergence, enabling efficient computation and mapping to a vector space for classifier input.
• Introduce two core properties for bag-to-class divergences: consistency under class-conditional independence and sensitivity to class-specific distribution shifts.
• Extend these properties to include robustness to sparse training sets, particularly in low-instance-per-bag scenarios.
• Use the resulting divergences as features in a downstream classifier, bridging the bag space (BS) and embedded space (ES) paradigms.

Experimental results

Research questions

• RQ1 Can modeling bags as probability distributions improve classification performance in multi-instance learning under data sparsity?
• RQ2 How does bag-to-class divergence compare to bag-to-bag divergence in terms of accuracy and computational efficiency?
• RQ3 What properties should a bag-to-class divergence satisfy to ensure robustness and statistical validity?
• RQ4 To what extent can class-conditional divergences like cKL handle sparse training sets with few instances per bag?
• RQ5 How does the proposed method generalize across diverse MI learning problems beyond the standard MI assumption?

Key findings

• The bag-to-class divergence approach significantly improves classification performance on sparse multi-instance datasets, particularly in cases with few instances per bag.
• The proposed class-conditional Kullback-Leibler (cKL) divergence outperforms standard bag-to-bag KL divergence in terms of misclassification rate under sparsity.
• The method reduces computational cost compared to full bag-to-bag divergence computation while maintaining high accuracy.
• Importance sampling enables efficient approximation of divergences, making the approach scalable to real-world MI problems.
• Theoretical properties of the divergence ensure robustness and allow for data-independent evaluation of MI methods.
• Empirical results on Musk1 and Musk2 datasets show that the method performs competitively with state-of-the-art MI methods despite sparse training sets.