[论文解读] Augmenting Scholarly Reading with Cross-Media Annotations

Qi Xu, Beat Signer|VUBIR (Vrije Universiteit Brussel)|Mar 18, 2026

Digital Humanities and Scholarship被引用 0

一句话总结

该论文设计了一个跨媒介注释工具，将 PDF 高亮与外部资源（音频、视频、网页）连接起来，以丰富学术阅读，描述架构、交互设计及潜在收益。

ABSTRACT

Scholarly reading often involves engaging with various supplementary materials beyond PDFs to support understanding. In practice, scholars frequently incorporate such external materials into their reading workflow through annotation. However, most existing PDF annotation tools support only a limited range of media types for embedding annotations in PDF documents. This paper investigates cross-media annotation as a design space for augmenting academic reading. We present a design exploration of a cross-media annotation tool that allows scholars to easily link PDF content with other documents and materials such as audio, video or web pages. The proposed design has the potential to enrich reading practices and enable scholars to guide and support other researchers' reading experiences.

研究动机与目标

• Motivate the need for cross-media annotations beyond PDFs in scholarly reading.
• Propose a general, extensible design for linking PDF content to diverse external resources.
• Demonstrate interaction patterns (drag-and-drop) to create and view cross-media annotations in situ.
• Outline an architecture that supports resource, selector, and link abstractions for cross-media annotations.
• Discuss the potential benefits for initial reading, rereading, and collaborative annotation.

提出的方法

• Present a design exploration of a cross-media annotation tool integrated with a PDF viewer.
• Ground the design in two principles: seamless linking from PDF content to external resources and easy access to those resources during reading.
• Describe a representative usage scenario to illustrate creating and browsing cross-media annotations via drag-and-drop interactions.
• Extend the existing cross-media annotation framework (RSL metamodel) to PDF-based scholarly reading, including Resources, Selectors, and Links.
• Propose a frontend/backend architecture with a PDF rendering overlay, plug-in interfaces to external apps, and a REST backend exposing RSL concepts.
• Suggest a margin-based, proximity-aware rendering and color-coding strategy to associate annotations with highlights.

Figure 1. The central PDF viewer serves as the primary space for reading and annotation, while external materials consulted during reading are displayed on both sides (web pages and videos on the left, other PDF documents on the right). Scholars can highlight content in the main PDF that requires ad

实验结果

研究问题

• RQ1 How can cross-media annotations be generically linked to PDF content and a variety of external resources (audio, video, web) during scholarly reading?
• RQ2 What architectural and interaction patterns enable in-situ rendering and management of cross-media annotations without duplicating content?
• RQ3 How can one-to-many cross-media annotations be supported in a scalable, user-friendly way?
• RQ4 What are the design benefits of cross-media annotations for initial reading, rereading, and sharing annotations with others?
• RQ5 How can the approach be extended with AI-assisted recommendations in future work?

主要发现

• A cross-media annotation tool can connect PDF highlights to diverse external materials via drag-and-drop, producing in-situ, margin-based pop-ups.
• The design supports one-to-many links, allowing a single highlight to reference multiple resources (e.g., a web page and a video).
• An architecture based on the RSL metamodel (Resource, Selector, Link) enables non-duplicative, transcluded content across resources.
• A combined visual encoding strategy using color hue and spatial proximity helps users identify associations between highlights and linked materials.
• The approach benefits initial reading, rereading, and collaborative use by organizing, recalling, and sharing context-rich annotations.

Figure 2. Cross-media annotation tool architecture