[Paper Review] Social Reader Perusall -- a Highly Effective Tool and Source of Formative Assessment Data

Jozef Hanč, Martina Hančová+1 more|arXiv (Cornell University)|Aug 14, 2023

Online Learning and AnalyticsComputer Science3 citations

TL;DR

This paper evaluates Perusall, an AI-powered social annotation platform, as a transformative tool for formative assessment and active learning in STEM education. By enabling asynchronous, collaborative annotation of digital materials and generating detailed analytics, Perusall enhances student engagement, supports personalized instruction, and significantly improves learning outcomes in blended and flipped classrooms, especially during the pandemic.

ABSTRACT

The contribution provides a detailed exploration of the online platform Perusall as an advanced social annotation technology in teaching and learning STEM disciplines. This exploration is based on the authors' insights and experiences from three years of implementing Perusall at P.J. Šafárik University in Košice, Slovakia. While the concept of social annotation technology and its educational applications are not novel, Perusall's advanced features, including AI and data science reports, enable its use in both synchronous and asynchronous blended and flipped learning environments. In this context, Perusall serves as a digital tool for collecting formative data, monitoring student progress, and identifying areas of difficulty. This assessment data can be effectively utilized in preparing and personalizing subsequent face-to-face group interactions, thereby enhancing and improving the learning experience. From a pedagogical viewpoint, Perusall's role was particularly significant during the Covid-19 pandemic, enabling effective, continuous, and engaging learning amidst social distancing and physical restrictions. Today, Perusall has become a key tool in blended learning, facilitating higher-order cognitive processes during the educational process and, with its multifaceted applications, serves as a modern catalyst in redefining educational experiences and outcomes.

Motivation & Objective

• To evaluate the effectiveness of Perusall as a social annotation tool in enhancing student engagement and formative assessment in STEM education.
• To explore how Perusall supports blended and flipped learning models through automated data collection and analytics.
• To demonstrate the platform’s role in maintaining educational continuity during the Covid-19 pandemic.
• To integrate data science tools (Python, Pandas, Selenium) for automated processing and visualization of student annotation data.
• To assess Perusall’s utility in preparing for face-to-face sessions by identifying student misconceptions and activity patterns.

Proposed method

• The authors implemented Perusall across multiple STEM courses at P.J. Šafárik University in Košice from 2020 to 2022, using it for asynchronous, collaborative annotation of digital content.
• Student annotations—comments and questions—were collected and analyzed using custom Jupyter notebooks built with Python, Pandas, and Selenium for automated data extraction and processing.
• Data visualization techniques, including boxplots and swarmplots, were used to represent student activity, engagement levels, and distribution of annotations by type (comments vs. questions).
• The platform’s AI-driven analytics provided insights into student understanding, identifying high-impact discussion threads and problematic content areas.
• The team used Perusall in a variety of educational contexts, including peer instruction, lab preparation, and science competition training.
• The study leveraged open-source data science tools to ensure reproducibility and scalability of data analysis workflows.

Figure 1: A view of the online platform Perusall during the Perusall Exchange 2022 conference. At the top, we see the types of contributions [ 1 ] , and at the bottom, we see a conference contribution in the form of a poster [ 6 ] .

Experimental results

Research questions

• RQ1 How does Perusall support formative assessment in blended and flipped STEM learning environments?
• RQ2 To what extent does Perusall enhance student engagement and collaboration in asynchronous online learning?
• RQ3 How can automated data analytics from student annotations inform and improve face-to-face teaching sessions?
• RQ4 What role did Perusall play in maintaining educational continuity during the Covid-19 pandemic?
• RQ5 How effective is Perusall in preparing students for science labs and academic competitions through collaborative annotation?

Key findings

• Perusall significantly enhanced student engagement by enabling asynchronous, collaborative annotation of digital content, with over 2,000 participants actively involved in the Perusall Exchange 2022 conference.
• The platform’s AI-powered analytics generated actionable formative assessment data, allowing instructors to identify student misconceptions and prioritize content in face-to-face sessions.
• Custom data pipelines using Python, Pandas, and Selenium enabled real-time processing and visualization of annotation data in milliseconds, reducing manual workload.
• The use of boxplots and swarmplots effectively visualized student activity, showing distinct patterns in comment and question distribution across different content types.
• Perusall proved essential during the pandemic, enabling continuous, interactive, and engaging learning despite physical distancing and institutional restrictions.
• The platform supported diverse educational applications, including peer instruction, lab preparation, and training for scientific competitions, demonstrating broad pedagogical versatility.

Figure 2: Perusall as a modern digital educational technology combining three digital technologies.