[Paper Review] Routing Networks and the Challenges of Modular and Compositional Computation

Clemens Rosenbaum, Ignacio Cases+2 more|arXiv (Cornell University)|Apr 29, 2019

Multimodal Machine Learning Applications77 references39 citations

TL;DR

This paper analyzes routing networks, a modular and compositional learning framework, focusing on training challenges that arise when learning both modules and their composition strategy.

ABSTRACT

Compositionality is a key strategy for addressing combinatorial complexity and the curse of dimensionality. Recent work has shown that compositional solutions can be learned and offer substantial gains across a variety of domains, including multi-task learning, language modeling, visual question answering, machine comprehension, and others. However, such models present unique challenges during training when both the module parameters and their composition must be learned jointly. In this paper, we identify several of these issues and analyze their underlying causes. Our discussion focuses on routing networks, a general approach to this problem, and examines empirically the interplay of these challenges and a variety of design decisions. In particular, we consider the effect of how the algorithm decides on module composition, how the algorithm updates the modules, and if the algorithm uses regularization.

Motivation & Objective

• Motivate modular and compositional computation as a solution to combinatorial complexity and dimensionality.
• Identify and articulate the key training challenges unique to routing networks.
• Analyze how module composition, module updates, and regularization affect learning.
• Provide empirical insights into design choices for routing, including training strategies and architectures.

Proposed method

• Describe routing networks as a general paradigm that jointly optimizes modules and their composition strategy.
• Review training strategies for routing, focusing on reinforcement learning and reparameterization approaches.
• Analyze design decisions such as optimization methods and router architecture.
• Empirically compare how different choices influence training stability, collapse, and overfitting.
• Discuss the lack of a unified formal framework and relate routing to coagent and Meta-MDP formalisms.

Experimental results

Research questions

• RQ1 What are the core training challenges that arise when learning both modular components and their routing strategy?
• RQ2 How do issues like training stability, module collapse, and overfitting manifest in routing networks?
• RQ3 What design choices in router training and architecture influence these challenges?
• RQ4 Can reinforcement learning and reparameterization provide principled training strategies for routing decisions?
• RQ5 What theoretical frameworks could better formalize modular and compositional learning?

Key findings

• Routing networks face non-stationarity from simultaneous updates to modules and routing policies.
• Module collapse occurs when the router overuses a subset of modules, hindering diversity and performance.
• Overfitting can be exacerbated by the router's ability to create highly local, specialized solutions.
• A unified formal framework for jointly training routers and modules is lacking, complicating convergence guarantees.
• Regularization, curriculum strategies, and meta-information-based routing can mitigate some instability and collapse.
• Empirical analysis highlights trade-offs between exploration, transfer, and interference in routing decisions.