[Paper Review] A Reality check of the benefits of LLM in business

Cheung Ming|arXiv (Cornell University)|Jun 9, 2024

ERP Systems Implementation and Impact6 citations

TL;DR

The paper empirically assesses the usefulness and readiness of large language models (LLMs) for core business processes by testing four accessible LLMs on real-world data, highlighting limitations in bias, context understanding, and prompt sensitivity.

ABSTRACT

Large language models (LLMs) have achieved remarkable performance in language understanding and generation tasks by leveraging vast amounts of online texts. Unlike conventional models, LLMs can adapt to new domains through prompt engineering without the need for retraining, making them suitable for various business functions, such as strategic planning, project implementation, and data-driven decision-making. However, their limitations in terms of bias, contextual understanding, and sensitivity to prompts raise concerns about their readiness for real-world applications. This paper thoroughly examines the usefulness and readiness of LLMs for business processes. The limitations and capacities of LLMs are evaluated through experiments conducted on four accessible LLMs using real-world data. The findings have significant implications for organizations seeking to leverage generative AI and provide valuable insights into future research directions. To the best of our knowledge, this represents the first quantified study of LLMs applied to core business operations and challenges.

Motivation & Objective

• Assess the usefulness and readiness of LLMs in business processes.
• Evaluate limitations of LLMs such as bias, contextual understanding, and prompt sensitivity.
• Conduct experiments with real data on multiple accessible LLMs to quantify capacities.
• Provide insights for organizations and directions for future research.

Proposed method

• Survey and discussion of common LLMs and their business use cases.
• Experiments using real data from four accessible LLMs to test usefulness and capacities.
• Bias analysis by testing LLMs’ ability to suggest references for project planning.
• Code generation experiments to generate SQL queries from natural language questions.
• Contextual prompt experiments to demonstrate the importance of supplying relevant context.
• Use of Poe as a wrapper to access multiple web-based LLMs and automate response collection.

Figure 1. Examples of a question on LLM with different prompts: (a) the question only; (b) write for sales, (c) write for data scientists.

Experimental results

Research questions

• RQ1 How useful are LLMs for planning, implementation, serving, and decision making in business contexts?
• RQ2 What are the major limitations of LLMs in business applications (bias, context understanding, prompt sensitivity)?
• RQ3 How well do LLMs perform in tasks like reference generation for literature reviews and SQL code generation from business questions?
• RQ4 To what extent does context and prompt design affect LLM outputs in business tasks?

Key findings

• Claude-instant yielded the highest average match for reference-generation accuracy (about 1.9 of 50) in the bias-related experiment.
• ChatGPT achieved an average of about 1.73 matched references, trailing Claude-instant.
• LLMs show better performance on single-table SQL queries than on joins requiring multi-table context.
• Prompt sensitivity significantly impacts outputs, underscoring the need for appropriate context in prompts.
• LLMs struggle with complex reasoning and context across tasks like coding and multi-table data analysis, suggesting limited readiness for some business applications.

Figure 2. Example of questions on different LLM: (a) ChatGPT; (b) Claude; (c) Llama; (d) PaLM