[論文レビュー] AI Governance and Accountability: An Analysis of Anthropic's Claude

Aman Priyanshu, Yash Maurya+1 more|arXiv (Cornell University)|May 2, 2024

Ethics and Social Impacts of AI被引用数 6

ひとこと要約

本論文はNISTとEU AI Actの枠組みを通じてAnthropicのClaudeを分析し、ガバナンスのギャップを特定し、透明性・ベンチマーク・データ処理の改善を含む緩和策を提案する。

ABSTRACT

As AI systems become increasingly prevalent and impactful, the need for effective AI governance and accountability measures is paramount. This paper examines the AI governance landscape, focusing on Anthropic's Claude, a foundational AI model. We analyze Claude through the lens of the NIST AI Risk Management Framework and the EU AI Act, identifying potential threats and proposing mitigation strategies. The paper highlights the importance of transparency, rigorous benchmarking, and comprehensive data handling processes in ensuring the responsible development and deployment of AI systems. We conclude by discussing the social impact of AI governance and the ethical considerations surrounding AI accountability.

研究の動機と目的

• Analyze Anthropic’s Claude using established AI governance frameworks (NIST AI Risk Management Framework and EU AI Act).
• Identify potential threats and risks posed by Claude and its Constitutional AI approach.
• Propose mitigation strategies for identified risks and governance gaps.
• Examine the implications of Constitutional AI on ethics, transparency, and accountability.
• Provide insights to inform broader AI governance discourse and practice.

提案手法

• Conduct threat analysis of Claude guided by NIST AI Risk Management Framework (Governance, Map, Measure, Manage) and EU AI Act risk categorization.
• Evaluate Anthropic’s Constitutional AI training approach and its ethical/practical implications.
• Compare privacy, data usage, and transparency practices against framework requirements and benchmarks.
• Propose concrete mitigations including transparency enhancements, benchmarking for hallucinations/bias, and data deletion/unlearning processes.
• Synthesize findings to discuss social impact and governance implications.

Figure 1: Anthropic’s Claude is one of the most popular large language model chatbots available to the everyday consumer. This paper presents a study of its practices and conduct through the lens of AI governance.

実験結果

リサーチクエスチョン

• RQ1 How does Anthropic’s Claude align with the NIST AI Risk Management Framework and the EU AI Act in terms of governance, risk mapping, and impact characterization?
• RQ2 What are the main threats and risks identified for Claude, including data privacy, hallucinations, third-party data usage, and constitutional AI ethics?
• RQ3 What mitigation strategies are proposed to address transparency, benchmarking, data handling, and accountability gaps?
• RQ4 What are the implications of Constitutional AI for fairness, bias, and cross-cultural ethical considerations?

主な発見

• Claude exhibits limited transparency in privacy policies, raising data usage and security concerns.
• There are concerns about hallucinations and biases in outputs, with limited open benchmarking data for independent validation.
• Third-party data usage and partner governance (e.g., with Google and Amazon) create accountability uncertainties.
• Anthropic’s Constitutional AI raises questions about universality of ethics and potential suppression of diverse perspectives.
• The analysis maps identified risks to NIST and EU framework categories, highlighting gaps in accountability and proactive risk management.
• Proposed mitigations emphasize transparency, rigorous bias/hallucination benchmarks, and robust data deletion/unlearning processes.

Figure 2: Rapidly growing customer visits on their Claude’s web interface