[Paper Review] Transferable Interactiveness Prior for Human-Object Interaction Detection.

Yong–Lu Li, Siyuan Zhou+6 more|arXiv (Cornell University)|Nov 20, 2018

Multimodal Machine Learning ApplicationsComputer Science27 references37 citations

TL;DR

This paper introduces a transferable interactiveness prior to improve Human-Object Interaction (HOI) detection by learning generalizable interaction patterns across datasets. It uses an Interactiveness Network to suppress non-interactions during inference, boosting performance across HICO-DET and V-COCO with state-of-the-art results.

ABSTRACT

Human-Object Interaction (HOI) Detection is an important problem to understand how humans interact with objects. In this paper, we explore extbf{Interactiveness Prior} which indicates whether human and object interact with each other or not. We found that interactiveness prior can be learned across HOI datasets, regardless of HOI category settings. Our core idea is to exploit an Interactiveness Network to learn the general interactiveness prior from multiple HOI datasets and perform Non-Interaction Suppression before HOI classification in inference. On account of the generalization of interactiveness prior, interactiveness network is a transferable knowledge learner and can be cooperated with any HOI detection models to achieve desirable results. We extensively evaluate the proposed method on HICO-DET and V-COCO datasets. Our framework outperforms state-of-the-art HOI detection results by a great margin, verifying its efficacy and flexibility. Source codes and models will be made publicly available.

Motivation & Objective

• To address the challenge of detecting human-object interactions in diverse and complex visual scenes.
• To learn a general interactiveness prior that captures whether a human and object are interacting, independent of specific HOI categories.
• To enable transferability of this prior across different HOI datasets and detection models.
• To improve HOI detection performance by suppressing non-interacting human-object pairs during inference.

Proposed method

• Train an Interactiveness Network on multiple HOI datasets to learn a general representation of interaction likelihood.
• Use the learned interactiveness prior to suppress non-interacting human-object pairs before final HOI classification.
• Integrate the Interactiveness Network as a plug-in module with any existing HOI detection model.
• Leverage multi-dataset supervision to enhance the generalization of the interactiveness prior.
• Apply non-interaction suppression during inference using the predicted interactiveness scores.
• Ensure the method is flexible and compatible with various HOI detection architectures.

Experimental results

Research questions

• RQ1 Can interactiveness prior be effectively learned and transferred across different HOI datasets?
• RQ2 Does learning a general interactiveness prior improve HOI detection performance on zero-shot or few-shot settings?
• RQ3 Can the interactiveness prior be used as a universal suppression mechanism across diverse HOI detection models?
• RQ4 How does the proposed method compare to state-of-the-art approaches on benchmark HOI datasets?

Key findings

• The proposed method achieves state-of-the-art performance on both HICO-DET and V-COCO datasets.
• The interactiveness prior generalizes well across different HOI datasets, even under varying category settings.
• The Interactiveness Network acts as a transferable knowledge learner, enhancing performance across multiple HOI detection models.
• Non-interaction suppression using the learned prior significantly improves detection accuracy by filtering out false positives.
• The method demonstrates strong flexibility and compatibility with existing HOI detection frameworks.
• Source code and trained models will be publicly released to support reproducibility and further research.