[Paper Review] A Conversationalist Approach to Information Quality in Information Interaction and Retrieval

Frans van der Sluis|arXiv (Cornell University)|Oct 13, 2022

Social Media and Politics4 citations

TL;DR

This paper proposes a conversationalist approach to information quality in search and retrieval, framing quality assessment as a collaborative, dialogic process among users rather than relying on expert or user feedback alone. It argues that quality is inherently interactive, shaped by social alignment and shared evaluation, and suggests that such conversations could improve ranking relevance and user engagement while challenging dominant authority and popularity biases.

ABSTRACT

Rather than using (proxies of) end user or expert judgment to decide on the ranking of information, this paper asks whether conversations about information quality might offer a feasible and valuable addition for ranking information. We introduce a theoretical framework for information quality, outlining how information interaction should be perceived as a conversation and quality be evaluated as a conversational contribution. Next, an overview is given of different systems of social alignment and their value for assessing quality and ranking information. We propose that a collaborative approach to quality assessment is preferable and raise key questions about the feasibility and value of such an approach for ranking information. We conclude that information quality is an inherently interactive concept, which involves an interaction between users of different backgrounds and in different situations as well as of quality signals on users' search behavior and experience.

Motivation & Objective

• To reframe information quality as a dynamic, interactive process rather than an intrinsic property of documents.
• To investigate whether collaborative, conversational assessments of quality could enhance ranking systems beyond current reliance on expert or user feedback.
• To examine the feasibility and value of social alignment mechanisms in evaluating and ranking information quality.
• To challenge the dominance of authority and popularity in search rankings by introducing interpretivist, dialogic models of quality.
• To explore how uncertainty and cognitive engagement might be leveraged through quality discussions in information interaction.

Proposed method

• Proposes a theoretical framework where information interaction is conceptualized as a conversation, and quality is evaluated as a conversational contribution.
• Draws on interpretivist epistemology to position quality as a shared, negotiated construct rather than an objective or purely subjective property.
• Reviews existing systems that incorporate social alignment—such as upvoting, flagging, and collaborative filtering—as precursors to conversational quality assessment.
• Analyzes the role of uncertainty and cognitive engagement in motivating user participation in quality discussions.
• Suggests that quality signals embedded in search result contexts can influence user affect and behavior, enhancing engagement.
• Proposes that future systems should integrate structured, collaborative quality conversations into ranking mechanisms, informed by crowd-sourced, situated assessments.

Experimental results

Research questions

• RQ1 Can information quality be meaningfully conceptualized as a conversational contribution rather than an intrinsic document property?
• RQ2 How might social alignment mechanisms in existing systems support or inform a conversationalist approach to quality assessment?
• RQ3 What are the conditions under which users would engage in collaborative quality discussions, and what motivates such engagement?
• RQ4 To what extent can a conversationalist approach reduce bias from authority and popularity in information retrieval?
• RQ5 How can quality conversations be designed to improve both ranking precision and user experience?

Key findings

• Information quality is not an intrinsic property but an interactive, negotiated construct shaped by diverse users in different contexts.
• Current ranking systems rely heavily on proxies of expert or user judgment, which can perpetuate existing power structures and biases.
• Users are strongly influenced by ranking position, but uncertainty about quality can drive cognitive engagement and information seeking.
• There is limited precedent for user engagement in quality discussions, though collective intelligence models suggest potential feasibility.
• A conversationalist approach could redirect focus from combating misinformation to understanding and promoting positive qualities of information.
• Scalability remains a challenge due to coverage limitations, cold start problems, and the need for large-scale, situated quality assessments.