[Paper Review] Experimenting a New Programming Practice with LLMs

Simiao Zhang, Jia-Ping Wang+4 more|arXiv (Cornell University)|Jan 2, 2024

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces AISD, an AI-aided software development framework that integrates human feedback throughout requirement analysis, system design, and validation phases. By iteratively refining use cases, system designs, and implementations using LLMs while keeping users actively involved, AISD achieves a 72.3% task pass rate on a novel benchmark, demonstrating that human-in-the-loop LLM development can significantly improve reliability and effectiveness in complex software tasks.

ABSTRACT

The recent development on large language models makes automatically constructing small programs possible. It thus has the potential to free software engineers from low-level coding and allow us to focus on the perhaps more interesting parts of software development, such as requirement engineering and system testing. In this project, we develop a prototype named AISD (AI-aided Software Development), which is capable of taking high-level (potentially vague) user requirements as inputs, generates detailed use cases, prototype system designs, and subsequently system implementation. Different from existing attempts, AISD is designed to keep the user in the loop, i.e., by repeatedly taking user feedback on use cases, high-level system designs, and prototype implementations through system testing. AISD has been evaluated with a novel benchmark of non-trivial software projects. The experimental results suggest that it might be possible to imagine a future where software engineering is reduced to requirement engineering and system testing only.

Motivation & Objective

• To address the limitations of fully automated LLM-based software development frameworks that lack user involvement.
• To improve the reliability and alignment of LLM-generated software by integrating human feedback during requirement analysis and system validation.
• To design a lightweight, user-friendly interface for requirement and design documents that supports both human and LLM understanding.
• To evaluate the effectiveness of an LLM-driven development pipeline that emphasizes iterative refinement with user input.
• To demonstrate that software engineering can be reduced to requirement engineering and system testing when supported by AI and human feedback.

Proposed method

• AISD takes high-level, vague user requirements as input and generates minimal, focused use cases and system design documents.
• The framework uses LLMs to iteratively refine use cases and system designs based on user feedback, ensuring alignment with stakeholder expectations.
• System design is decomposed into atomic coding tasks, which are automatically implemented using LLMs.
• Prototype implementations are tested via unit and system tests, and failures trigger feedback loops to revise requirements, design, or code.
• A novel benchmark with reference use cases is created to objectively evaluate task completion and alignment.
• The framework emphasizes minimal human involvement—only at key decision points—following a 'less is more' philosophy.

Experimental results

Research questions

• RQ1 Can an LLM-based software development framework achieve high task success rates while minimizing human involvement?
• RQ2 How does integrating user feedback during requirement analysis and system validation improve the quality and correctness of generated software?
• RQ3 To what extent can a human-in-the-loop approach outperform fully automated LLM frameworks in complex software development tasks?
• RQ4 How effective is a lightweight, use-case-driven design process in guiding accurate and maintainable code generation?
• RQ5 Can iterative testing and feedback loops significantly reduce the number of failed implementations in LLM-based development?

Key findings

• AISD achieved a 72.3% task pass rate on a novel benchmark of non-trivial software projects, demonstrating strong performance in complex task completion.
• The framework significantly outperformed existing end-to-end LLM-based approaches by incorporating user feedback at critical stages of requirement and design refinement.
• By keeping users engaged only at key decision points, AISD reduced token consumption while maintaining high accuracy and reliability.
• The use of standardized, minimal-use-case documents improved both human and LLM comprehension and reduced ambiguity in system specification.
• Iterative testing and feedback loops enabled early detection of mismatches between requirements and implementations, reducing rework and improving alignment.
• The results suggest that future software development may be reduced to requirement engineering and system testing when supported by AI and human-in-the-loop refinement.