[Paper Review] Deformable Capsules for Object Detection

Rodney Lalonde, Naji Khosravan+1 more|arXiv (Cornell University)|Apr 11, 2021

Advanced Neural Network Applications4 citations

TL;DR

This paper introduces DeformCaps, the first capsule network for object detection, combining a novel SplitCaps architecture and SE-Routing algorithm to enable efficient, scalable one-stage detection on MS COCO. It achieves state-of-the-art performance on par with CNN-based detectors while reducing false positives and improving generalization to unusual object poses.

ABSTRACT

Capsule networks promise significant benefits over convolutional networks by storing stronger internal representations, and routing information based on the agreement between intermediate representations' projections. Despite this, their success has been limited to small-scale classification datasets due to their computationally expensive nature. Though memory efficient, convolutional capsules impose geometric constraints that fundamentally limit the ability of capsules to model the pose/deformation of objects. Further, they do not address the bigger memory concern of class-capsules scaling up to bigger tasks such as detection or large-scale classification. In this study, we introduce a new family of capsule networks, deformable capsules ( extit{DeformCaps}), to address a very important problem in computer vision: object detection. We propose two new algorithms associated with our extit{DeformCaps}: a novel capsule structure ( extit{SplitCaps}), and a novel dynamic routing algorithm ( extit{SE-Routing}), which balance computational efficiency with the need for modeling a large number of objects and classes, which have never been achieved with capsule networks before. We demonstrate that the proposed methods efficiently scale up to create the first-ever capsule network for object detection in the literature. Our proposed architecture is a one-stage detection framework and it obtains results on MS COCO which are on par with state-of-the-art one-stage CNN-based methods, while producing fewer false positive detection, generalizing to unusual poses/viewpoints of objects.

Motivation & Objective

• To address the lack of capsule networks in large-scale object detection, a critical gap in computer vision.
• To overcome the computational and representational limitations of traditional capsule networks in scaling to datasets like MS COCO.
• To enable capsules to model complex object poses and deformations by removing rigid spatial constraints in routing.
• To design a memory-efficient capsule architecture capable of handling large numbers of object classes and instances.
• To demonstrate that capsule networks can achieve performance comparable to CNNs on one-stage detection benchmarks.

Proposed method

• Introduces DeformCaps, a one-stage object detection framework using capsules for joint localization and classification.
• Proposes SplitCaps, a new capsule structure that decouples class-agnostic instantiation parameters from class-specific presence and identity, reducing dimensionality and enabling scalability.
• Designs SE-Routing, a Squeeze-and-Excitation-inspired dynamic routing algorithm that maximizes agreement between child capsule projections without iterative loops.
• Employs deformable sampling in parent capsules to adaptively attend to child capsules, eliminating fixed local receptive fields and improving geometric flexibility.
• Uses a capsule-based head with reconstruction head for mask prediction, enabling potential single-shot instance segmentation.
• Leverages capsule vector representations to encode pose, deformation, hue, and texture, enhancing part-whole relationship modeling.

Experimental results

Research questions

• RQ1 Can capsule networks be scaled to large-scale object detection tasks such as MS COCO?
• RQ2 Can a capsule-based architecture outperform or match CNN-based one-stage detectors in accuracy while reducing false positives?
• RQ3 Can deformable routing mechanisms improve generalization to unusual object poses and viewpoints?
• RQ4 Can a capsule architecture avoid the computational explosion of global routing while maintaining representational power?
• RQ5 Can a novel capsule structure like SplitCaps eliminate the dimensional blow-up that previously hindered large-scale training?

Key findings

• DeformCaps achieves object detection performance on MS COCO that is on-par with state-of-the-art one-stage CNN-based detectors.
• The model produces significantly fewer false positive detections compared to baseline methods like CenterNet.
• Qualitative results show improved generalization to unusual object poses and viewpoints, supporting the theoretical advantage of capsules in pose robustness.
• SplitCaps successfully reduces the dimensionality of capsule representations, enabling training on large-scale datasets for the first time in capsule networks.
• SE-Routing enables efficient dynamic routing without iterative loops, improving computational efficiency while maintaining routing agreement.
• The proposed framework is the first capsule-based object detection system in the literature, marking a foundational step toward scalable capsule applications in vision.