[Paper Review] Generative Artificial Intelligence in Higher Education: Evidence from an Analysis of Institutional Policies and Guidelines

Nora McDonald, Aditya Johri+2 more|arXiv (Cornell University)|Jan 12, 2024

AI and HR Technologies40 citations

TL;DR

The paper analyzes GenAI policies from 116 US R1 universities, showing a majority encourage GenAI use and many provide classroom guidance, syllabi, and DEI-focused ethics, highlighting potential policy burden on faculty.

ABSTRACT

The release of ChatGPT in November 2022 prompted a massive uptake of generative artificial intelligence (GenAI) across higher education institutions (HEIs). HEIs scrambled to respond to its use, especially by students, looking first to regulate it and then arguing for its productive integration within teaching and learning. In the year since the release, HEIs have increasingly provided policies and guidelines to direct GenAI. In this paper we examined documents produced by 116 US universities categorized as high research activity or R1 institutions to comprehensively understand GenAI related advice and guidance given to institutional stakeholders. Through an extensive analysis, we found the majority of universities (N=73, 63%) encourage the use of GenAI and many provide detailed guidance for its use in the classroom (N=48, 41%). More than half of all institutions provided sample syllabi (N=65, 56%) and half (N=58, 50%) provided sample GenAI curriculum and activities that would help instructors integrate and leverage GenAI in their classroom. Notably, most guidance for activities focused on writing, whereas code and STEM-related activities were mentioned half the time and vaguely even when they were (N=58, 50%). Finally, more than one half of institutions talked about the ethics of GenAI on a range of topics broadly, including Diversity, Equity and Inclusion (DEI) (N=60, 52%). Overall, based on our findings we caution that guidance for faculty can become burdensome as extensive revision of pedagogical approaches is often recommended in the policies.

Motivation & Objective

• Assess how US R1 higher education institutions address generative AI in policies and guidelines.
• Characterize the prevalence of GenAI encouragement, classroom guidance, syllabi resources, and curriculum materials.
• Evaluate the extent of ethical considerations, including DEI, in GenAI guidance.
• Identify potential implications for teaching practice and pedagogy revision prompted by policy recommendations.

Proposed method

• Systematic analysis of policy documents and guidelines from 116 US universities classified as high research activity (R1).
• Quantitative coding of policy features such as encouragement of GenAI use, classroom guidance, sample syllabi, GenAI curriculum/activities, and ethics coverage.
• Aggregation of counts and proportions to summarize prevalence of policy elements.
• Descriptive reporting of which GenAI activities are emphasized (e.g., writing versus coding/STEM).
• Extraction of DEI and ethics-related topics within the guidance.

Experimental results

Research questions

• RQ1 To what extent do R1 universities encourage GenAI use in their policies and guidelines?
• RQ2 How prevalent are classroom guidance, sample syllabi, and GenAI curriculum materials across institutions?
• RQ3 What topics are most frequently addressed in ethics or DEI guidance related to GenAI?
• RQ4 What are common focal areas for GenAI activities within policy guidance (writing vs. coding/STEM)?

Key findings

• 63% of universities (n=73) encourage the use of GenAI.
• 41% (n=48) provide detailed classroom guidance for GenAI use.
• 56% (n=65) provide sample syllabi.
• 50% (n=58) provide sample GenAI curriculum and activities for instructors.
• Ethics guidance on GenAI covers a range of topics, including DEI, with 52% (n=60) addressing such issues.
• Guidance often emphasizes writing activities and may underrepresent coding and STEM-oriented uses.