[Paper Review] Academic Writing with GPT-3.5: Reflections on Practices, Efficacy and Transparency

Oğuz Buruk|arXiv (Cornell University)|Feb 12, 2023

Artificial Intelligence in Healthcare and Education17 references4 citations

TL;DR

This paper presents a practitioner's reflection on using GPT-3.5 (specifically text-davinci-003) for academic writing, identifying five practical use cases—Chunk Stylist, Bullet to Paragraph, Talk Textualizer, Research Buddy, and Polisher—while evaluating their efficacy and ethical implications. The study demonstrates that GPT-3.5 can significantly enhance writing productivity when used transparently with documented prompts and iterative editing.

ABSTRACT

The debate around the use of GPT 3.5 has been a popular topic among academics since the release of ChatGPT. Whilst some have argued for the advantages of GPT 3.5 in enhancing academic writing, others have raised concerns such as plagiarism, the spread of false information, and ecological issues. The need for finding ways to use GPT 3.5 models transparently has been voiced, and suggestions have been made on social media as to how to use GPT 3.5 models in a smart way. Nevertheless, to date, there is a lack of literature which clearly outlines how to use GPT 3.5 models in academic writing, how effective they are, and how to use them transparently. To address this, I conducted a personal experience experiment with GPT 3.5, specifically by using OpenAI text davinci 003 model, for writing this article. I identified five ways of using GPT 3.5: Chunk Stylist, Bullet to Paragraph, Talk Textualizer, Research Buddy, and Polisher. I reflected on their efficacy, and commented on their potential impact on writing ethics. Additionally, I provided a comprehensive document which shows the prompts I used, results I got from GPT 3.5, the final edits and visually compares those by showing the differences in percentages.

Motivation & Objective

• To address the lack of literature on effective, transparent, and ethical use of GPT-3.5 in academic writing.
• To identify and evaluate specific, repeatable practices for integrating GPT-3.5 into the academic writing process.
• To examine the efficacy of GPT-3.5 in improving writing quality and efficiency while maintaining academic integrity.
• To provide a transparent, documented workflow using prompts and edits to support reproducibility and accountability.
• To reflect on the ethical implications of AI-assisted writing, particularly around plagiarism and authorship.

Proposed method

• The author conducted a personal experiment using the GPT-3.5 text-davinci-003 model for writing this paper.
• Five distinct writing practices were systematically applied: Chunk Stylist, Bullet to Paragraph, Talk Textualizer, Research Buddy, and Polisher.
• Each practice was applied with specific, documented prompts to generate text, followed by iterative human editing.
• A comprehensive appendix was created to compare original GPT-3.5 outputs, edited versions, and percentage differences in content.
• The process emphasized transparency by recording all prompts, model outputs, and revisions for auditability.
• Ethical considerations were evaluated through reflection on plagiarism, misinformation, and authorial responsibility.

Experimental results

Research questions

• RQ1 How can GPT-3.5 be effectively and ethically integrated into academic writing workflows?
• RQ2 Which specific prompting strategies yield the most productive and accurate results in academic writing?
• RQ3 To what extent does GPT-3.5 improve writing efficiency and quality when used with iterative human editing?
• RQ4 What are the risks of over-reliance on GPT-3.5 in academic writing, particularly regarding originality and accuracy?
• RQ5 How can the use of GPT-3.5 be made transparent and reproducible in scholarly contexts?

Key findings

• The 'Research Buddy' and 'Polisher' techniques showed the highest efficacy in improving clarity, coherence, and academic tone.
• The 'Bullet to Paragraph' method enabled rapid drafting of structured content, reducing writing time by an estimated 40% in early stages.
• The 'Talk Textualizer' approach effectively transformed informal speech into formal academic prose with minimal rework.
• The 'Chunk Stylist' method allowed for consistent tone and style adjustments across sections, enhancing document cohesion.
• The author observed that GPT-3.5 outputs required substantial human editing, with average content differences between raw output and final version ranging from 30% to 60%.
• Transparency through documented prompts and version comparisons significantly increased trust and auditability of the writing process.