[Paper Review] Measuring Conversational Productivity in Child Forensic Interviews

Victor Ardulov, Manoj Kumar+3 more|arXiv (Cornell University)|Jun 8, 2018

Language, Discourse, Communication Strategies8 references3 citations

TL;DR

This paper proposes a computational framework to measure conversational productivity in child forensic interviews using topic modeling and lexical entrainment to assess how effectively children respond to interview goals. By constructing a dynamic interview agenda and scoring responses for alignment and responsiveness, the method outperforms traditional word count metrics in capturing information relevance and shows stronger robustness to age-related language variation.

ABSTRACT

Child Forensic Interviewing (FI) presents a challenge for effective information retrieval and decision making. The high stakes associated with the process demand that expert legal interviewers are able to effectively establish a channel of communication and elicit substantive knowledge from the child-client while minimizing potential for experiencing trauma. As a first step toward computationally modeling and producing quality spoken interviewing strategies and a generalized understanding of interview dynamics, we propose a novel methodology to computationally model effectiveness criteria, by applying summarization and topic modeling techniques to objectively measure and rank the responsiveness and conversational productivity of a child during FI. We score information retrieval by constructing an agenda to represent general topics of interest and measuring alignment with a given response and leveraging lexical entrainment for responsiveness. For comparison, we present our methods along with traditional metrics of evaluation and discuss the use of prior information for generating situational awareness.

Motivation & Objective

• To develop an objective, computable metric for assessing verbal productivity in child forensic interviews, moving beyond subjective or simplistic measures.
• To address the limitations of traditional metrics like word count and human-coded 'richness' by introducing a topic-modeling-based approach to evaluate information relevance and alignment with interview goals.
• To model the interview agenda dynamically using topic modeling to represent the interviewer's information objectives and assess how well the child's responses align with these goals.
• To incorporate lexical entrainment as a proxy for responsiveness and rapport, reflecting trust and engagement in dyadic interactions.
• To create a scalable, automated evaluation system that reduces reliance on human labeling while improving consistency and fidelity in assessing interview quality.

Proposed method

• Construct a dynamic interview agenda using topic modeling (e.g., LDA or trigram-based models) on prior transcripts to represent key topics the interviewer aims to elicit.
• Score each child’s utterance for 'agenda alignment' by measuring lexical overlap between the response and the current agenda topic, using TF-IDF or similar weighting.
• Measure 'responsiveness' via lexical entrainment—quantifying how closely the child’s vocabulary matches that of the interviewer in real time, reflecting rapport and engagement.
• Combine agenda alignment and responsiveness into a composite productivity score using weighted parameters β and γ to balance relevance and engagement.
• Normalize scores across sessions and use value-iteration or decision-theoretic models to guide adaptive interviewing strategies.
• Validate the model using 527 forensic interview transcripts, comparing metrics against age, word count, and human-coded benchmarks.

Experimental results

Research questions

• RQ1 Can topic modeling and lexical entrainment be used to objectively measure the productivity of child responses in forensic interviews?
• RQ2 How does the proposed productivity metric compare to traditional word count and human-coded richness in capturing information relevance?
• RQ3 To what extent does the model’s performance vary with the child’s age, and is it robust to developmental differences in language ability?
• RQ4 Can the agenda-based framework be used to generate real-time, adaptive interview strategies that optimize information retrieval?
• RQ5 How well do the proposed metrics correlate with known indicators of rapport and effective interviewing?

Key findings

• The proposed productivity metric, combining agenda alignment and responsiveness, demonstrated a statistically significant but weaker correlation with child age (r = 0.25) compared to word count (r = 0.46), indicating reduced sensitivity to developmental language variation.
• The agenda-based model successfully filtered out low-relevance utterances, producing a sparser but more informative signal than word count, which exhibited high volatility and low fidelity.
• Lexical entrainment emerged as a strong proxy for responsiveness, with high alignment to known trust indicators in dyadic interactions, supporting its use as a rapport measure.
• The model’s use of a trigram-based topic model achieved strong performance, suggesting that even simple NLP techniques can yield meaningful insights in high-stakes forensic contexts.
• The composite productivity score (π*) showed higher consistency and lower variance across age groups than word count, indicating greater robustness and reliability.
• The framework enables real-time, adaptive scoring and can be used to pre-define 'gold standard' agendas, supporting the development of evidence-based interviewing policies.