[論文レビュー] Towards Stable and Efficient Training of Verifiably Robust Neural Networks

研究の動機と目的

• Motivate the need for verifiable robustness guarantees in DNNs and analyze limitations of existing linear-relaxation and IBP methods.
• Introduce CROWN-IBP as a hybrid certified training method that blends IBP and CROWN bounds.
• Demonstrate that CROWN-IBP is both computationally efficient and yields tighter robustness bounds during training.
• Show substantial improvements over prior IBP and linear-relaxation baselines on MNIST and CIFAR-10 under l-infinity attacks.

提案手法

• Develop a training objective that combines natural cross-entropy loss with a robust loss based on a bound that is a weighted combination of IBP and CROWN-IBP margins.
• Perform a forward bound pass using IBP to obtain tight per-layer z bounds; apply a ReLU relaxation to create linear upper/lower bounds (Eq. 10–11).
• Perform a backward bound pass in a CROWN-like manner starting from the last layer to obtain a tight lower bound (Eq. 12–13).
• Merge the forward IBP bounds and the backward CROWN-IBP bounds to compute a composite margin bound underline{m}(x,ε) used in the robust loss.
• Allow a tunable parameter β to balance IBP and CROWN-IBP contributions, and a κ parameter to trade off standard versus verified accuracy ( Eq. 9 ).
• Achieve improved scalability by avoiding full-CROWN complexity and leveraging the small output dimension n_L for efficient backward propagation.]
• research_questions: ["Can a hybrid bound combining IBP and CROWN-IBP provide both stability during training and tighter verifiable robustness bounds?","Does CROWN-IBP enable significantly better verified error rates than pure IBP or linear-relaxation methods across MNIST and CIFAR-10 for common l_infinity budgets?","What is the impact of κ and β scheduling on the trade-off between standard accuracy and verifiable robustness?","Is the proposed method scalable to larger networks without prohibitive computational or memory overhead?"]
• key_findings: ["CROWN-IBP yields lower verified error than IBP alone across MNIST and CIFAR-10 for tested ε settings (e.g., MNIST at ε=0.3: 7.02% verified error vs 8.21% for IBP; CIFAR-10 at ε=2/255: 46.03% verified error vs 55.88% for IBP).","CROWN-IBP outperforms pure IBP in both standard (clean) and verified accuracy, and achieves state-of-the-art results among IBP- and linear-relaxation-based methods on the evaluated datasets.","The method maintains favorable efficiency, with backward bound propagation complexity scaled by the output size n_L rather than the network width, improving scalability over traditional CROWN/Convex Adversarial Polytope approaches.","CROWN-IBP enables flexible trading between robustness and accuracy via κ and β schedules, demonstrating improved Pareto fronts for standard vs verified accuracy.","The study reports verifiable bounds for CIFAR-10 at ε=16/255, showing the method's applicability to larger perturbation budgets and datasets."]
• table_headers: []
• table_rows: []

関連論文