[Paper Review] Automated Optical Multi-layer Design via Deep Reinforcement Learning

Haozhu Wang, Zeyu Zheng+2 more|arXiv (Cornell University)|Jun 21, 2020

Advanced optical system design37 references4 citations

TL;DR

This paper proposes OML-PPO, a deep reinforcement learning framework that frames multi-layer optical design as a sequence generation problem, using proximal policy optimization with a custom recurrent network to discover high-performance thin-film structures. It outperforms human-designed and state-of-the-art heuristic designs in two energy-relevant tasks: achieving 97.64% average absorption in a 5-layer ultra-wideband absorber and 16.60 visible light enhancement in a 42-layer incandescent filter, surpassing a memetic algorithm by 8.5%.

ABSTRACT

Optical multi-layer thin films are widely used in optical and energy applications requiring photonic designs. Engineers often design such structures based on their physical intuition. However, solely relying on human experts can be time-consuming and may lead to sub-optimal designs, especially when the design space is large. In this work, we frame the multi-layer optical design task as a sequence generation problem. A deep sequence generation network is proposed for efficiently generating optical layer sequences. We train the deep sequence generation network with proximal policy optimization to generate multi-layer structures with desired properties. The proposed method is applied to two energy applications. Our algorithm successfully discovered high-performance designs, outperforming structures designed by human experts in task 1, and a state-of-the-art memetic algorithm in task 2.

Motivation & Objective

• To address the inefficiency and sub-optimality of human-expert-driven multi-layer optical film design in large design spaces.
• To develop a scalable, learning-based method that does not rely on hand-crafted heuristics for combinatorial optical design.
• To enable automatic discovery of near-optimal multi-layer structures with unknown optimal layer count.
• To improve design performance in energy applications such as broadband absorbers and incandescent light filters.
• To validate the method’s effectiveness through benchmark comparisons with human-designed and state-of-the-art heuristic algorithms.

Proposed method

• The multi-layer optical design task is reformulated as a sequence generation problem, where each layer’s material and thickness are generated step-by-step.
• A deep recurrent neural network with non-repetitive gating and auto-regressive generation is proposed to model the sequence generation process and improve exploration.
• Proximal Policy Optimization (PPO) is used to train the agent, with rewards shaped based on target optical performance (e.g., absorption or reflectivity).
• The reward function encourages high performance over specified wavelength ranges, such as average absorption in [400, 2000] nm or visible light enhancement factor.
• The method is applied to two real-world energy applications: ultra-wideband absorbers and incandescent light bulb filters with tailored spectral responses.
• An ablation study evaluates the contribution of non-repetitive gating and auto-regressive generation to model convergence and performance.

Experimental results

Research questions

• RQ1 Can deep reinforcement learning effectively solve multi-layer optical design as a sequence generation problem without prior knowledge of the optimal number of layers?
• RQ2 How do non-repetitive gating and auto-regressive generation improve the performance and convergence of the RL agent in optical design?
• RQ3 Can the proposed method discover higher-performance optical structures than human experts and state-of-the-art heuristic algorithms in real-world energy applications?
• RQ4 To what extent does the model generalize across different design complexities, such as varying numbers of layers?
• RQ5 Does the integration of domain-specific architectural inductive biases (e.g., non-repetitive gating) lead to more efficient exploration and better final designs?

Key findings

• OML-PPO discovered a 5-layer ultra-wideband absorber with 97.64% average absorption across 400–2000 nm, outperforming a human-designed structure with 95.37% average absorption.
• The method discovered a 14-layer absorber achieving 99.24% average absorption, approaching near-perfect performance.
• For incandescent light bulb filters, OML-PPO achieved a visible light enhancement factor of 16.60, which is 8.5% higher than a 41-layer structure designed by a state-of-the-art memetic algorithm.
• The ablation study showed that non-repetitive gating significantly improves convergence and final performance, outperforming models without it by 3.95% on average absorption.
• Combining both non-repetitive gating and auto-regressive generation yielded the best performance, with a mean absorption of 94.98% ± 0.99% across 10 training runs.
• The model demonstrated scalability and robustness, reliably discovering high-performance designs across complex, high-dimensional design spaces.