[Paper Review] Symmetry and Group in Attribute-Object Compositions

Yong–Lu Li, Yue Xu+2 more|arXiv (Cornell University)|Apr 1, 2020

Domain Adaptation and Few-Shot Learning39 references4 citations

TL;DR

This paper introduces symmetry as a fundamental principle in attribute-object composition, proposing SymNet—a group-theory-inspired framework that enforces symmetry, closure, associativity, identity, and invertibility through end-to-end training. It achieves state-of-the-art performance in compositional zero-shot learning by using Relative Moving Distance (RMD) to classify attributes based on transformation robustness rather than unstable visual patterns.

ABSTRACT

Attributes and objects can compose diverse compositions. To model the compositional nature of these general concepts, it is a good choice to learn them through transformations, such as coupling and decoupling. However, complex transformations need to satisfy specific principles to guarantee the rationality. In this paper, we first propose a previously ignored principle of attribute-object transformation: Symmetry. For example, coupling peeled-apple with attribute peeled should result in peeled-apple, and decoupling peeled from apple should still output apple. Incorporating the symmetry principle, a transformation framework inspired by group theory is built, i.e. SymNet. SymNet consists of two modules, Coupling Network and Decoupling Network. With the group axioms and symmetry property as objectives, we adopt Deep Neural Networks to implement SymNet and train it in an end-to-end paradigm. Moreover, we propose a Relative Moving Distance (RMD) based recognition method to utilize the attribute change instead of the attribute pattern itself to classify attributes. Our symmetry learning can be utilized for the Compositional Zero-Shot Learning task and outperforms the state-of-the-art on widely-used benchmarks. Code is available at https://github.com/DirtyHarryLYL/SymNet.

Motivation & Objective

• To identify and formalize symmetry as a previously overlooked principle in attribute-object transformations.
• To develop a unified deep learning framework that enforces group axioms and symmetry for robust attribute-object composition.
• To improve compositional zero-shot learning by leveraging transformation invariance and relative movement in latent space.
• To replace reliance on fragile visual attribute patterns with robust attribute change dynamics for classification.

Proposed method

• SymNet employs two subnetworks: a Coupling Network (CoN) for adding attributes and a Decoupling Network (DecoN) for removing them.
• The framework enforces group axioms—closure, associativity, identity, and invertibility—through dedicated loss functions during end-to-end training.
• Symmetry is embedded as a core objective: adding an attribute to an object that already has it, or removing one that it lacks, must leave the object unchanged.
• A novel Relative Moving Distance (RMD) metric computes the distance between input and transformed embeddings to infer attribute presence.
• The RMD mechanism compares distances after 'add' and 'remove' operations: closer output to input indicates the attribute is present.
• Attention mechanisms are used to enhance feature representation, and L2 distance is used as the primary metric for RMD computation.

Experimental results

Research questions

• RQ1 Can symmetry be formalized as a foundational principle in attribute-object composition, ensuring invariance under redundant transformations?
• RQ2 How can group theory be applied to design a differentiable, end-to-end deep learning framework for attribute-object composition?
• RQ3 Can RMD-based attribute recognition outperform traditional pattern-based methods by focusing on transformation dynamics rather than visual appearance?
• RQ4 To what extent do enforcing group axioms and symmetry improve generalization in compositional zero-shot learning?
• RQ5 Does the proposed method generalize across diverse benchmarks such as MIT-States and UT-Zappos?

Key findings

• SymNet achieves state-of-the-art performance on compositional zero-shot learning benchmarks, with 52.1% top-1 accuracy on UT-Zappos and 19.9% on MIT-States.
• Removing the symmetry loss (L_sym) leads to a 1.6% drop in top-1 accuracy on UT-Zappos and a 1.6% drop on MIT-States, confirming its critical role.
• The ablation study shows that removing the classification loss (L_cls) causes the most severe degradation, dropping to 28.7% top-1 on UT-Zappos, indicating the importance of joint optimization.
• Using L1 or cosine distance instead of L2 distance in RMD leads to significantly worse performance, with L1 dropping to 37.5% top-1 on UT-Zappos.
• Visualization via t-SNE confirms that all group axioms—including closure, invertibility, and commutativity—are well satisfied in the learned latent space.
• The symmetry property is robustly learned: objects with an attribute show minimal movement after 'add' and large movement after 'remove', and vice versa for objects without the attribute.