[Paper Review] Auditing Work: Exploring the New York City algorithmic bias audit regime

Lara Groves, Jacob Metcalf+3 more|arXiv (Cornell University)|Feb 12, 2024

Taxation and Compliance Studies6 citations

TL;DR

The paper analyzes NYC Local Law 144, the first municipal regime requiring independent bias audits for AEDTs in hiring, via interviews with 16 auditors to assess practicality, governance, and policy implications; it finds the regime flawed due to vague definitions, industry lobbying, access barriers, and divergent auditor roles.

ABSTRACT

In July 2023, New York City (NYC) initiated the first algorithm auditing system for commercial machine-learning systems. Local Law 144 (LL 144) mandates NYC-based employers using automated employment decision-making tools (AEDTs) in hiring to undergo annual bias audits conducted by an independent auditor. This paper examines lessons from LL 144 for other national algorithm auditing attempts. Through qualitative interviews with 16 experts and practitioners within the regime, we find that LL 144 has not effectively established an auditing regime. The law fails to clearly define key aspects, such as AEDTs and independent auditors, leading auditors, AEDT vendors, and companies using AEDTs to define the law's practical implementation in ways that failed to protect job applicants. Contributing factors include the law's flawed transparency-driven theory of change, industry lobbying narrowing the definition of AEDTs, practical and cultural challenges faced by auditors in accessing data, and wide disagreement over what constitutes a legitimate auditor, resulting in four distinct 'auditor roles.' We conclude with four recommendations for policymakers seeking to create similar bias auditing regimes, emphasizing clearer definitions, metrics, and increased accountability. By exploring LL 144 through the lens of auditors, our paper advances the evidence base around audit as an accountability mechanism, providing guidance for policymakers seeking to create similar regimes.

Motivation & Objective

• Evaluate the practical components of bias audits under LL 144 in NYC.
• Examine relational dynamics and incentives among auditors, employers, AEDT vendors, and regulators.
• Characterize auditors' experiences to inform broader policy and practice on algorithmic audits.
• Assess how LL 144 shapes accountability mechanisms for high-stakes hiring decisions.

Proposed method

• Conduct qualitative interviews with 16 experts and practitioners involved in LL 144 audits.
• Apply grounded theory to identify latent themes and coding from interview transcripts.
• Use Atlas.ti for coding, starting from an initial 102-code codebook and refining to 16 codes.
• Analyze public comments and audit artefacts to contextualize interview findings.

Experimental results

Research questions

• RQ1 RQ1: What are the practical components of a bias audit in this context?
• RQ2 RQ2: What are the relational dynamics and incentives that make for an effective bias auditing regime?
• RQ3 RQ3: What are the experiences of auditors, and how can they inform wider policy and practice?

Key findings

• Auditing under LL 144 is primarily about managing stakeholder-relational dynamics created by the law’s accountability structures.
• There is no single, agreed-upon auditor role; four distinct auditor ‘actor’ categories emerge, offering different services and creating fragmentation.
• There is strong disagreement over what constitutes a legitimate auditor and auditing practice, with debates over independence and scope.
• Auditors face significant practical and cultural barriers to data access from employers and AEDT vendors, affecting audit quality and feasibility.
• Public availability of audit reports is limited in the initial months, hindering transparency and accountability.
• The law’s transparency-driven theory of change is viewed as insufficient to stop biased AEDTs, and industry lobbying shaped the law’s scope to narrow the definition of AEDTs.