[Paper Review] RobOT: Robustness-Oriented Testing for Deep Learning Systems

Jingyi Wang, Jialuo Chen+5 more|arXiv (Cornell University)|Feb 11, 2021

Adversarial Robustness in Machine Learning49 references4 citations

TL;DR

This paper proposes RobOT, a robustness-oriented testing framework for deep learning systems that uses a novel metric, FOL (Feature-wise Outlier Likelihood), to guide test case generation and retraining. By prioritizing test cases that most improve model robustness, RobOT achieves a 67.02% increase in adversarial robustness—50.65% higher than the state-of-the-art method, DeepGini.

ABSTRACT

Recently, there has been a significant growth of interest in applying software engineering techniques for the quality assurance of deep learning (DL) systems. One popular direction is deep learning testing, where adversarial examples (a.k.a.~bugs) of DL systems are found either by fuzzing or guided search with the help of certain testing metrics. However, recent studies have revealed that the commonly used neuron coverage metrics by existing DL testing approaches are not correlated to model robustness. It is also not an effective measurement on the confidence of the model robustness after testing. In this work, we address this gap by proposing a novel testing framework called Robustness-Oriented Testing (RobOT). A key part of RobOT is a quantitative measurement on 1) the value of each test case in improving model robustness (often via retraining), and 2) the convergence quality of the model robustness improvement. RobOT utilizes the proposed metric to automatically generate test cases valuable for improving model robustness. The proposed metric is also a strong indicator on how well robustness improvement has converged through testing. Experiments on multiple benchmark datasets confirm the effectiveness and efficiency of RobOT in improving DL model robustness, with 67.02% increase on the adversarial robustness that is 50.65% higher than the state-of-the-art work DeepGini.

Motivation & Objective

• Address the gap that existing deep learning testing methods, which rely on neuron coverage metrics, show little correlation with actual model robustness.
• Develop a testing framework that integrates model retraining with test case generation to ensure robustness improvements are measurable and effective.
• Propose a lightweight, robustness-correlated metric (FOL) that quantifies the value of each test case in improving model robustness.
• Enable convergence detection of robustness improvement by using the proposed metric as a convergence indicator.
• Provide an end-to-end solution for enhancing model robustness against adversarial examples in safety-critical applications.

Proposed method

• Introduce FOL (Feature-wise Outlier Likelihood) as a novel metric to quantify how much a test case contributes to improving model robustness.
• Use FOL to guide a fuzzing-based test case generation strategy that prioritizes inputs most likely to expose and improve model vulnerabilities.
• Integrate iterative retraining into the testing pipeline, where models are retrained using adversarial examples generated based on high-FOL test cases.
• Leverage FOL as a convergence metric to determine when robustness improvement has plateaued, reducing unnecessary retraining cycles.
• Design a feedback loop where test case generation and model retraining are iteratively refined based on FOL scores to maximize robustness gains.

Experimental results

Research questions

• RQ1 To what extent does the proposed FOL metric correlate with actual model robustness improvement compared to traditional neuron coverage metrics?
• RQ2 Can FOL-guided test case generation produce more effective adversarial examples than coverage-based approaches in improving model robustness?
• RQ3 How does RobOT’s robustness improvement compare to state-of-the-art methods like DeepGini and ADAPT in terms of adversarial robustness gain?
• RQ4 Can the FOL metric effectively indicate convergence in robustness improvement, thus reducing redundant testing and retraining?

Key findings

• RobOT achieved a 67.02% increase in adversarial robustness across benchmark datasets, significantly outperforming existing methods.
• The improvement from RobOT was 50.65% higher than that of the state-of-the-art method, DeepGini, demonstrating superior effectiveness.
• FOL-guided test case generation discovered more valuable adversarial examples than ADAPT, leading to a 39.67% average robustness gain compared to ADAPT’s 24.79%.
• The FOL metric strongly correlates with actual robustness improvement, enabling reliable convergence detection during testing.
• Experiments confirmed that test cases with high FOL scores consistently led to greater robustness gains upon retraining, validating the metric’s predictive power.