[Paper Review] ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognition

Swapnil Parekh|arXiv (Cornell University)|Feb 28, 2026

Phonetics and Phonology Research0 citations

TL;DR

ACES audits an ASR model by extracting an accent-discriminative subspace, stress-testing with subspace-constrained attacks, and evaluating a project-out intervention to assess coupling between accent information and recognition. Results show accent-subspace attacks amplify WER disparity across seven accents, while simple erasure worsens performance.

ABSTRACT

ASR systems exhibit persistent performance disparities across accents, but whether these gaps reflect superficial biases or deep structural vulnerabilities remains unclear. We introduce ACES, a three-stage audit that extracts accent-discriminative subspaces from ASR representations, constrains adversarial attacks to them, and tests whether removing them improves fairness. On Wav2Vec2-base with seven accents, imperceptible perturbations (~60 dB SNR) along the accent subspace amplify the WER disparity gap by nearly 50% (21.3->31.8 pp), exceeding random-subspace controls; a permuted-label test confirms specificity to genuine accent structure. Partially removing the subspace worsens both WER and disparity, revealing that accent-discriminative and recognition-critical features are deeply entangled. ACES thus positions accent subspaces as powerful fairness-auditing tools, not simple erasure levers.

Motivation & Objective

• Motivate the need to understand whether WER disparities across accents reflect deep structural vulnerabilities rather than superficial biases.
• Propose ACES, a three-stage audit to extract accent-subspaces, stress-test with constrained attacks, and evaluate interventions.
• Quantify how accent-discriminative directions in representations relate to recognition performance and fairness.

Proposed method

• Extract an accent subspace from Wav2Vec2-base representations using mean-pooled layer embeddings and a ridge probe to maximize accent discrimination.
• Constrain adversarial attacks to move representations along the accent subspace by optimizing a combined loss: CTC loss plus a subspace-alignment term with L2 budget ε.
• Evaluate perturbations under four conditions (clean, unconstrained PGD, random-subspace, accent-subspace) and measure coupling m(x) and WER disparity.
• Perform a project-out intervention by partially removing the accent subspace at inference to test causal coupling with disparity.
• Validate subspace via probe accuracy, stability (principal angles), and correlation between projection and WER; use permuted-label control as specificity check.

Figure 1: ACES: subspace extraction, stress-test (waveform PGD, L2 $\varepsilon$ ), and project-out. Conceptually, the representation $\mathbf{h}$ is projected onto the plane spanned by $\mathbf{U}$ ; coupling $m(x)$ measures how much the attack moves $\mathbf{h}$ along that subspace.

Experimental results

Research questions

• RQ1 Does perturbing along accent-discriminative directions exacerbate WER disparities across accents beyond random subspaces?
• RQ2 Is removing the accent subspace at inference able to reduce disparity, indicating a causal coupling between accent information and recognition errors?
• RQ3 Are there signs of entanglement between accent-discriminative features and recognition-critical cues in ASR representations?
• RQ4 How specific is the accent-subspace effect to genuine accent structure versus data-derived artefacts?
• RQ5 What are the practical implications of ACES for auditing fairness and avoiding unsafe linear erasure interventions?

Key findings

• Accent-subspace attacks amplify WER disparity from 21.3 to 31.8 percentage points, exceeding random-subspace controls.
• Mean coupling m(x) for accent-subspace attacks is about twice that of random-subspace attacks, indicating preferential activation of accent directions.
• Permuted-label control eliminates the accent-subspace advantage, confirming specificity to real accent structure.
• Partial project-out (α=0.5) worsens clean WER and disparity, indicating that accent-discriminative and recognition-critical features are entangled and erasure is not a reliable fairness fix.
• Accent geometry suggests decodability is higher in early layers (2–4) and attenuates in deeper layers, guiding where accent information is encoded.
• Across seven accents, six show positive accent-subspace advantage under attack; the effect is strongest for Indian, Bermuda, and Malaysia, with smaller effects for US and Wales.

Figure 2: Three-track diagnostic (1-column): probe accuracy, corr(projection, WER), stability (principal angle, °) vs. layer. Layer 3 maximizes probe accuracy while maintaining stability below 50°; $k{=}8$ (dashed).