[Paper Review] The Role of Normware in Trustworthy and Explainable AI

Giovanni Sileno, Alexander Boer+1 more|arXiv (Cornell University)|Dec 6, 2018

Ethics and Social Impacts of AI6 references8 citations

TL;DR

This paper introduces 'normware'—computational artifacts specifying norms—as a critical, ecologically integrated component for building trustworthy and explainable AI. By embedding normative constraints directly into the decision-making cycle, normware enables systems to check for unintended consequences, align with ethical standards, and generate explainable justifications, offering a novel architectural layer beyond software and hardware.

ABSTRACT

For being potentially destructive, in practice incomprehensible and for the most unintelligible, contemporary technology is setting high challenges on our society. New conception methods are urgently required. Reorganizing ideas and discussions presented in AI and related fields, this position paper aims to highlight the importance of normware--that is, computational artifacts specifying norms--with respect to these issues, and argues for its irreducibility with respect to software by making explicit its neglected ecological dimension in the decision-making cycle.

Motivation & Objective

• To address growing concerns about unintended, unethical, or opaque outcomes in AI systems, especially in autonomous and human-interactive contexts.
• To argue that current AI development overlooks the need for a dedicated computational layer—normware—that explicitly encodes norms and ethical constraints.
• To position normware as an irreducible, ecologically embedded component that guides but does not fully control system behavior.
• To propose a new architectural paradigm where normware functions as a dynamic, context-sensitive check on decisions, improving trust and explainability.
• To bridge gaps between AI ethics, legal norms, and technical implementation by grounding normative reasoning in system design.

Proposed method

• Proposes normware as a distinct computational artifact that specifies norms, functioning as a check-and-balance mechanism within the decision-making cycle.
• Introduces a multi-layered system architecture where normware components interact with software, simulators, and user interfaces to validate decisions.
• Uses a 'strategic driver' mechanism that evaluates plans against high-level norms (e.g., ecological sustainability, fairness), rejecting non-compliant options.
• Employs a 'second-order oracle' to generate training data that satisfies neutrality constraints, reducing statistical bias in machine learning models.
• Applies alignment checking between prediction and justification modules, ensuring explanations are compatible with expert knowledge models.
• Models normware as ecologically coexisting with other system components, drawing analogies to epistemic and legal pluralism in normative systems.

Experimental results

Research questions

• RQ1 How can norms be formally encoded as computational artifacts to guide autonomous AI decisions in a way that is both interpretable and enforceable?
• RQ2 What architectural role does normware play in preventing unintended consequences in AI systems, especially when human oversight is reduced?
• RQ3 How can normware support explainability by enabling systems to generate justifications that align with expert or societal knowledge?
• RQ4 In what ways does normware differ from traditional software and hardware, and why is it irreducible to existing abstraction layers?
• RQ5 Can normware be implemented in non-symbolic AI systems (e.g., deep learning), and how would it interface with such components?

Key findings

• Normware provides a necessary, ecologically embedded layer that complements software by embedding normative constraints directly into the decision-making process.
• The integration of normware enables proactive detection of unintended consequences through strategic checks, such as rejecting ecologically harmful plans in a simulated environment.
• A second-order oracle can be used to generate training data that satisfies neutrality constraints, thereby reducing statistical bias in models.
• Explainability is enhanced by decoupling prediction and justification modules, where the justification component validates the prediction against expert knowledge.
• Normware supports ontological alignment not through logical equivalence, but through functional compatibility in specific input contexts, enabling flexibility across domains.
• Even in non-symbolic AI systems, normware can function via symbolic interfaces, as norms are communicated and interpreted through language and structured representations.