[论文解读] The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces

Kyle Lo, Joseph Chee Chang+53 more|arXiv (Cornell University)|Mar 25, 2023

AI in Service Interactions被引用 7

一句话总结

语义阅读器项目开发面向学术PDF的AI驱动互动阅读界面，推出十个原型、对300+参与者的可用性研究，以及一个生产就绪的阅读器，以提高研究论文的发现、效率、理解、综合和可获取性。

ABSTRACT

Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the need for new technology to support the reading process grows. In contrast to the process of finding papers, which has been transformed by Internet technology, the experience of reading research papers has changed little in decades. The PDF format for sharing research papers is widely used due to its portability, but it has significant downsides including: static content, poor accessibility for low-vision readers, and difficulty reading on mobile devices. This paper explores the question "Can recent advances in AI and HCI power intelligent, interactive, and accessible reading interfaces -- even for legacy PDFs?" We describe the Semantic Reader Project, a collaborative effort across multiple institutions to explore automatic creation of dynamic reading interfaces for research papers. Through this project, we've developed ten research prototype interfaces and conducted usability studies with more than 300 participants and real-world users showing improved reading experiences for scholars. We've also released a production reading interface for research papers that will incorporate the best features as they mature. We structure this paper around challenges scholars and the public face when reading research papers -- Discovery, Efficiency, Comprehension, Synthesis, and Accessibility -- and present an overview of our progress and remaining open challenges.

研究动机与目标

• Address the challenges readers face with scholarly PDFs across discovery, efficiency, comprehension, synthesis, and accessibility.
• Investigate how AI and HCI can create intelligent, interactive, and accessible reading interfaces atop legacy PDFs.
• Develop and evaluate multiple prototype interfaces to enhance the reading experience for researchers.
• Provide production-ready tools and open resources to enable broader adoption and continued research in scholarly reading

提出的方法

• Develop ten research prototypes of AI-powered interactive reading interfaces for papers (e.g., CiteSee, CiteRead, Scim, Ocean, ScholarPhi, Paper Plain, Papeo, Threddy, Relatedly, SciA11y).
• Conduct usability studies with 300+ participants to assess reading experience improvements.
• Integrate with Semantic Scholar and open science resources to support discovery and collaboration.
• Leverage layout-aware document parsing and large language models to access and augment PDF content.
• Provide an open production reader and ongoing features as maturation progresses.
• Present a structured view of five reading challenges and map prototypes to them

Figure 1. The Semantic Reader Project consists of research, product, and open science resources. The Semantic Reader product 1 is a free interactive interface for research papers. It supports standard reading features (e.g., (A) table of contents), integration with Semantic Scholar (e.g., (B) save t

实验结果

研究问题

• RQ1 Can AI-powered interactive reading interfaces significantly improve discovery, efficiency, comprehension, synthesis, and accessibility for scholarly papers?
• RQ2 How effectively can legacy PDFs be converted into dynamic, accessible representations that support diverse reading tasks?
• RQ3 What roles do inline citations, annotations, and multimodal explanations play in enhancing scholarly reading workflows?
• RQ4 How can researchers synthesize related work across many papers using interactive tools to form coherent overviews?
• RQ5 What are the open research opportunities at the intersection of AI and HCI for future scholarly reading interfaces?

主要发现

• Ten research prototypes were developed to address core reading challenges and demonstrate potential benefits.
• Usability studies involving 300+ participants indicate improved reading experiences across the targeted tasks.
• A production Semantic Reader interface has been developed and will incorporate new features as they mature.
• Prototype work demonstrates progress in discovery (CiteSee, CiteRead), efficient navigation (Scim, Ocean), in-situ explanations (ScholarPhi, Paper Plain, Papeo), and synthesis (Threddy, Relatedly).
• The project emphasizes open science resources to enable broader adoption and ongoing research in AI-enabled scholarly reading interfaces.
• The effort highlights the feasibility of augmenting legacy PDFs with intelligent, interactive reading features rather than requiring new document formats.

Figure 2. CiteSee (Chang et al . , 2023 ) highlights citations to familiar papers (e.g., recently read or saved in their libraries) as well as unfamiliar papers to help readers avoid overlooking important citations when conducting literature reviews. Clicking on Expand surfaces additional context, s