[Paper Review] Learning the Pareto Front with Hypernetworks

Aviv Navon, Aviv Shamsian+2 more|arXiv (Cornell University)|Oct 8, 2020

Robot Manipulation and Learning62 references30 citations

TL;DR

The paper introduces Pareto HyperNetworks (PHNs) to learn the entire Pareto front of multi-objective problems with a single unified model, enabling inference-time selection of any preferred operating point.

ABSTRACT

Multi-objective optimization (MOO) problems are prevalent in machine learning. These problems have a set of optimal solutions, called the Pareto front, where each point on the front represents a different trade-off between possibly conflicting objectives. Recent MOO methods can target a specific desired ray in loss space however, most approaches still face two grave limitations: (i) A separate model has to be trained for each point on the front; and (ii) The exact trade-off must be known before the optimization process. Here, we tackle the problem of learning the entire Pareto front, with the capability of selecting a desired operating point on the front after training. We call this new setup Pareto-Front Learning (PFL). We describe an approach to PFL implemented using HyperNetworks, which we term Pareto HyperNetworks (PHNs). PHN learns the entire Pareto front simultaneously using a single hypernetwork, which receives as input a desired preference vector and returns a Pareto-optimal model whose loss vector is in the desired ray. The unified model is runtime efficient compared to training multiple models and generalizes to new operating points not used during training. We evaluate our method on a wide set of problems, from multi-task regression and classification to fairness. PHNs learn the entire Pareto front at roughly the same time as learning a single point on the front and at the same time reach a better solution set. Furthermore, we show that PHNs can scale to generate large models like ResNet18. PFL opens the door to new applications where models are selected based on preferences that are only available at run time.

Motivation & Objective

• Define Pareto Front Learning (PFL) as learning a model that, at inference, can produce Pareto-optimal solutions for any given objective preference.
• Develop Pareto HyperNetworks (PHNs) that generate target network weights conditioned on a preference vector.
• Provide two PHN training variants (PHN-LS and PHN-EPO) to map preferences to Pareto-optimal solutions.
• Demonstrate scalability to large models (e.g., ResNet18) and show runtime and performance benefits over per-ray baseline methods.

Proposed method

• Formulate PHN as a hypernetwork h(r;φ) that outputs target network weights θ_r conditioned on a m-dimensional preference vector r on the simplex.
• Train PHN by sampling r from a Dirichlet distribution and applying either linear scalarization (PHN-LS) or Exact Pareto Optimal (PHN-EPO) updates.
• PHN-LS optimizes the weighted sum ∑i r_i ℓ_i via standard gradient descent.
• PHN-EPO uses the EPO descent direction to move toward a Pareto-optimal point on the ray defined by r, combining gradients convexly.
• Discuss scalability strategies including partial Parameter Generation, hypernetwork normalization layers, and chunking to manage large target networks.
• Compare against baselines (LS, PMTL, EPO, CPMTL) and evaluate using Hypervolume (HV) and Uniformity metrics.

Experimental results

Research questions

• RQ1 Can a single hypernetwork generate Pareto-optimal models for arbitrary preferences on the Pareto front?
• RQ2 Do PHNs provide complete front coverage and exact Pareto optimality with EPO updates?
• RQ3 How do PHN-LS and PHN-EPO compare to per-ray baselines in terms of HV and uniformity across tasks?
• RQ4 Is PHN scalable to large architectures (e.g., ResNet18) without proportional parameter growth?
• RQ5 What runtime advantages do PHNs offer over training many separate models for different preferences?

Key findings

• PHN learns the entire Pareto front in a single model and enables inference-time preference selection.
• PHN-EPO achieves continuous front coverage and exact Pareto optimal points on queried rays, while PHN-LS provides fast approximations.
• PHN delivers superior HV and uniformity across multi-task classification, fairness, segmentation, and regression tasks compared to LS, PMTL, and EPO baselines.
• PHN significantly reduces training time and scales to large networks like ResNet18 when using chunking and parameter-sharing strategies.
• PHN maintains or improves front coverage while reducing the number of trainable parameters versus per-ray baselines by substantial factors.
• PHN demonstrates faster runtimes (up to an order of magnitude) while achieving competitive or better HV across evaluated datasets.