[Paper Review] A Reference Architecture for Designing Foundation Model based Systems

Qinghua Lu, Liming Zhu+3 more|arXiv (Cornell University)|Apr 13, 2023

Scientific Computing and Data ManagementDecision Sciences3 citations

TL;DR

This paper proposes a pattern-oriented reference architecture for designing responsible foundation model (FM)-based systems, addressing challenges like architectural evolution, accountability, and trustworthiness. It introduces seven key design patterns across system, operation, and supply chain layers to enable adaptability, modifiability, and responsible AI practices—validated through mapping to a real-world RAI chatbot system using GPT-4 and retrieval-augmented generation (RAG).

ABSTRACT

The release of ChatGPT, Gemini, and other large language model has drawn huge interests on foundations models. There is a broad consensus that foundations models will be the fundamental building blocks for future AI systems. However, there is a lack of systematic guidance on the architecture design. Particularly, the the rapidly growing capabilities of foundations models can eventually absorb other components of AI systems, posing challenges of moving boundary and interface evolution in architecture design. Furthermore, incorporating foundations models into AI systems raises significant concerns about responsible and safe AI due to their opaque nature and rapidly advancing intelligence. To address these challenges, the paper first presents an architecture evolution of AI systems in the era of foundation models, transitioning from "foundation-model-as-a-connector" to "foundation-model-as-a-monolithic architecture". The paper then identifies key design decisions and proposes a pattern-oriented reference architecture for designing responsible foundation-model-based systems. The patterns can enable the potential of foundation models while ensuring associated risks.

Motivation & Objective

• Address the lack of systematic architectural guidance for foundation model (FM)-based AI systems amid rapid capability growth.
• Tackle architectural challenges such as moving boundaries and evolving interfaces due to FMs absorbing external components.
• Ensure responsible AI through accountability, traceability, and trustworthiness in FM-integrated systems.
• Provide a future-proof, adaptable architecture that supports evolving FM capabilities and regulatory compliance.
• Develop a pattern-oriented reference architecture that embeds responsible AI by design across system, operation, and supply chain layers.

Proposed method

• Proposes a three-stage architecture evolution: from traditional AI systems to FM-as-connector, and finally to FM-as-monolithic architecture.
• Introduces seven key design patterns: black box recorder, standardised reporter, verifier, think-aloud, agent team, co-versioning registry, and AIBOM registry.
• Applies the think-aloud pattern to enhance transparency by exposing intermediate reasoning steps of FMs.
• Uses the black box recorder to log inputs, outputs, and intermediate steps with timestamps and locations for auditability and accountability.
• Employs the verifier pattern to allow human experts to review and correct FM-generated outputs.
• Integrates supply chain controls via AIBOM registry and co-versioning registry to track third-party AI components and their responsible AI (RAI) credentials.

Experimental results

Research questions

• RQ1 How does the architectural role of foundation models evolve from coordination connectors to monolithic systems?
• RQ2 What are the key design decisions and trade-offs in building responsible FM-based systems with respect to adaptability, modularity, and trustworthiness?
• RQ3 How can accountability and traceability be systematically supported in FM-based systems involving multiple stakeholders?
• RQ4 What architectural patterns enable the integration of responsible AI practices such as auditability, explainability, and risk monitoring?
• RQ5 To what extent can the proposed reference architecture be mapped to and validated in real-world FM-based systems?

Key findings

• The reference architecture successfully maps to a real-world responsible AI (RAI) chatbot system using GPT-4 and RAG, demonstrating its practical applicability.
• The system layer includes foundation models, interaction components, and data sources, while the operation and supply chain layers ensure runtime and procurement traceability.
• The black box recorder enables full logging of inputs, outputs, and intermediate steps with timestamps and locations, supporting accountability and auditability.
• The verifier pattern allows human experts to review and edit FM-generated responses, enhancing reliability and trustworthiness.
• The AIBOM registry and co-versioning registry enable traceability of third-party components, including their responsible AI metrics and verifiable credentials.
• The architecture supports modifiability and adaptability, allowing future integration of fine-tuned or sovereign FMs and evolving system requirements.