[Paper Review] AI in Software Engineering: Case Studies and Prospects

Lei Wang|arXiv (Cornell University)|Sep 27, 2023

IoT and Edge/Fog ComputingComputer Science3 citations

TL;DR

This paper analyzes IBM Watson and Google AlphaGo as case studies to explore how AI techniques—particularly deep learning and reinforcement learning—enhance software engineering by enabling intelligent systems. It finds that Watson supports human decision-making through knowledge-based analysis, while AlphaGo achieves self-directed decision-making via neural networks and self-play, demonstrating AI's potential to solve complex real-world problems beyond human capability.

ABSTRACT

Artificial intelligence (AI) and software engineering (SE) are two important areas in computer science. In recent years, researchers are trying to apply AI techniques in various stages of software development to improve the overall quality of software products. Moreover, there are also some researchers focus on the intersection between SE and AI. In fact, the relationship between SE and AI is very weak; however, methods and techniques in one area have been adopted in another area. More and more software products are capable of performing intelligent behaviour like human beings. In this paper, two cases studies which are IBM Watson and Google AlphaGo that use different AI techniques in solving real world challenging problems have been analysed, evaluated and compared. Based on the analysis of both case studies, using AI techniques such as deep learning and machine learning in software systems contributes to intelligent systems. Watson adopts 'decision making support' strategy to help human make decisions; whereas AlphaGo uses 'self-decision making' to choose operations that contribute to the best outcome. In addition, Watson learns from man-made resources such as paper; AlphaGo, on the other hand, learns from massive online resources such as photos. AlphaGo uses neural networks and reinforcement learning to mimic human brain, which might be very useful in medical research for diagnosis and treatment. However, there is still a long way to go if we want to reproduce human brain in machine and view computers as thinkers, because human brain and machines are intrinsically different. It would be more promising to see whether computers and software systems will become more and more intelligent to help with real world challenging problems that human beings cannot do.

Motivation & Objective

• To examine the integration of AI techniques in software engineering to improve software quality and development efficiency.
• To analyze real-world applications of AI in complex problem-solving through case studies of IBM Watson and Google AlphaGo.
• To compare different AI strategies—decision-making support versus self-decision making—in intelligent software systems.
• To evaluate the role of learning sources (man-made vs. online data) in training AI systems for software applications.
• To explore the future potential of AI in software engineering for solving problems beyond human capacity.

Proposed method

• Conducted comparative analysis of IBM Watson and Google AlphaGo as representative AI systems in software engineering.
• Evaluated Watson’s use of DeepQA, which combines multiple algorithms to assess source reliability, time, and geography for question answering.
• Analyzed AlphaGo’s architecture using deep neural networks and Monte Carlo tree search for policy and value networks in Go gameplay.
• Contrasted learning mechanisms: Watson trained on curated human-authored knowledge (e.g., documents), while AlphaGo learned from vast self-play and game data.
• Used qualitative and comparative evaluation to assess decision-making autonomy and learning strategies in both systems.
• Explored implications of neural network-based AI for future software systems in domains like medical diagnosis and autonomous systems.

Experimental results

Research questions

• RQ1 How do AI techniques such as deep learning and reinforcement learning enhance software systems in real-world applications?
• RQ2 What are the differences in decision-making strategies between IBM Watson (decision support) and Google AlphaGo (self-decision making)?
• RQ3 How do the learning sources (curated human knowledge vs. massive online data) affect the performance and adaptability of AI systems?
• RQ4 To what extent can AI systems mimic human intelligence in complex, high-stakes domains like gaming and medical diagnosis?
• RQ5 What are the limitations of current AI systems in replicating human brain functionality, and what are the implications for future software engineering?

Key findings

• IBM Watson uses a decision-making support strategy, analyzing information from curated human-made sources like documents to generate confidence scores and features for answers.
• Google AlphaGo employs self-decision making through deep neural networks and reinforcement learning, enabling it to master the game of Go without human intervention.
• AlphaGo learns from massive online data, including game photos and self-play games, allowing it to discover novel strategies beyond human expertise.
• The integration of deep learning and reinforcement learning in software systems significantly enhances their ability to make intelligent, adaptive decisions.
• Despite progress, fundamental differences between human brains and machines—such as parallelism and self-organization—limit the ability to fully replicate human cognition in software.
• AI techniques in software engineering show strong potential for solving complex, real-world problems that are infeasible for humans to address alone, especially in domains like medical diagnosis and autonomous systems.