[Paper Review] Neural topology optimization: the good, the bad, and the ugly

Suryanarayanan Manoj Sanu, Alejandro M. Aragón+1 more|arXiv (Cornell University)|Jul 19, 2024

Piezoelectric Actuators and ControlEngineering3 citations

TL;DR

This paper investigates neural topology optimization (neural TO), where neural networks reparameterize the design space to reshape the objective landscape in topology optimization. It reveals that neural networks can enhance exploration in non-convex problems but introduce non-convexities in convex ones, slowing convergence—highlighting the method's potential ('the good'), limitations in smooth landscapes ('the bad'), and the challenge of architecture/hyperparameter selection ('the ugly').

ABSTRACT

Neural networks (NNs) hold great promise for advancing inverse design via topology optimization (TO), yet misconceptions about their application persist. This article focuses on neural topology optimization (neural TO), which leverages NNs to reparameterize the decision space and reshape the optimization landscape. While the method is still in its infancy, our analysis tools reveal critical insights into the NNs' impact on the optimization process. We demonstrate that the choice of NN architecture significantly influences the objective landscape and the optimizer's path to an optimum. Notably, NNs introduce non-convexities even in otherwise convex landscapes, potentially delaying convergence in convex problems but enhancing exploration for non-convex problems. This analysis lays the groundwork for future advancements by highlighting: 1) the potential of neural TO for non-convex problems and dedicated GPU hardware (the "good"), 2) the limitations in smooth landscapes (the "bad"), and 3) the complex challenge of selecting optimal NN architectures and hyperparameters for superior performance (the "ugly").

Motivation & Objective

• To analyze the impact of neural network reparameterization on the optimization landscape in topology optimization.
• To identify when neural TO improves convergence and when it hinders it, particularly in convex versus non-convex problems.
• To evaluate the role of neural network architecture and hyperparameters in shaping optimization trajectories and performance.
• To provide diagnostic tools for understanding how neural networks alter objective and loss landscapes in inverse design.
• To guide future research by categorizing neural TO into three distinct challenges: the good, the bad, and the ugly.

Proposed method

• Neural TO redefines decision variables from discrete density fields to trainable neural network parameters, which output physical densities within the design domain.
• A finite element analysis evaluates the objective (e.g., structural compliance) for each network-generated density field.
• The optimizer updates the network’s weights and biases using gradients of the objective with respect to the network parameters.
• The study compares neural TO using three architectures—CNN, SIREN, and MLP—across benchmark problems (MBB, bridge, tensile) with varying material volume constraints.
• Performance is quantified using three metrics: objective value, converged iteration count, and thresholded compliance, all normalized against a baseline density-based TO with MMA.
• Visualization of objective landscapes, optimization trajectories, and loss surfaces is used to analyze the impact of reparameterization on convergence behavior.

Experimental results

Research questions

• RQ1 How does neural network reparameterization affect the convexity and structure of the objective landscape in topology optimization?
• RQ2 In what scenarios does neural TO improve convergence speed or solution quality compared to standard density-based TO?
• RQ3 Why does neural TO sometimes fail to converge faster or even degrade performance in smooth, convex problems?
• RQ4 How do different neural network architectures (e.g., SIREN, MLP, CNN) influence the optimization path and final solution quality?
• RQ5 What are the dominant implicit biases introduced by neural networks that affect the optimization trajectory in inverse design?

Key findings

• Neural TO can introduce non-convexities into otherwise convex objective landscapes, potentially delaying convergence in convex problems.
• For non-convex problems, neural TO enhances exploration and can reach superior solutions, such as the global minimum in the two-bar problem, which standard MMA missed.
• SIREN networks achieved the best performance in the two-bar problem, converging to the global minimum at (1,0) with a 1.67× reduction in convergence iterations compared to baseline MMA.
• In the MBB problem with 0.3 volume fraction, the SIREN network achieved a 99% objective value relative to baseline, while converging in 0.31× the iterations, indicating faster convergence.
• For the tensile problem at 0.1 volume fraction, the MLP network achieved a 1.14× objective value but required 3.92× more iterations than baseline, indicating poor performance and instability.
• The choice of neural network architecture and hyperparameters significantly impacts performance, with no single architecture outperforming across all problems and constraints, underscoring the 'ugly' challenge of optimal selection.