[Paper Review] Finder: A Multimodal AI-Powered Search Framework for Pharmaceutical Data Retrieval

Suyash Mishra, Srikanth Patil+3 more|arXiv (Cornell University)|Jan 6, 2026

Biomedical Text Mining and Ontologies0 citations

TL;DR

Finder unifies multimodal pharmaceutical data retrieval using hybrid sparse and dense search, enhanced by LLM-based query optimization, and demonstrates enterprise-scale performance.

ABSTRACT

AI is transforming pharmaceutical search, where traditional systems struggle with multimodal content and manual curation. Finder is a scalable AI-powered framework that unifies retrieval across text, images, audio, and video using hybrid vector search, combining sparse lexical and dense semantic models. Its modular pipeline ingests diverse formats, enriches metadata, and stores content in a vector-native backend. Finder supports reasoning-aware natural language search, improving precision and contextual relevance. The system has processed over 291,400 documents, 31,070 videos, and 1,192 audio files in 98 languages. Techniques like hybrid fusion, chunking, and metadata-aware routing enable intelligent access across regulatory, research, and commercial domains.

Motivation & Objective

• Address the challenge of multimodal data volume and regulatory complexity in pharma search.
• Develop a modular ingestion and normalization pipeline for text, images, audio, and video.
• Enable hybrid lexical and semantic retrieval with LLM-assisted query understanding.
• Improve relevance, context, and retrieval speed through metadata-aware routing and ranking.

Proposed method

• Ingest diverse formats (PDF, Word, PPT, audio, video, images) and normalize content into structured text with metadata.
• Use a modular tagging pipeline with extractive and abstractive tagging, plus document summarization.
• Vectorize content with a fusion of BM42 lexical scores and Mixedbread dense embeddings for hybrid search.
• Apply Claude 4 for intent parsing and query optimization; perform dynamic reformulation and auto-filtering.
• Rank results via a weighted fusion of fuzzy similarity and normalized BM42 scores, with ongoing learning-to-rank development.

Figure 1: Finder Architecture Diagram

Experimental results

Research questions

• RQ1 How can multimodal pharmaceutical data be retrieved efficiently across text, images, audio, and video?
• RQ2 What is the impact of hybrid lexical-semantic retrieval on precision and recall in regulated pharma domains?
• RQ3 Can intent parsing and dynamic reformulation improve retrieval for vague or incomplete queries?
• RQ4 What are the latency and scalability characteristics of an enterprise-grade multimodal search system in pharma?
• RQ5 How does metadata-aware routing affect search relevance and user trust in regulated workflows?

Key findings

• Finder achieved an average relevance rate of 87.7% across 1,000 test queries.
• In enterprise-scale testing, end-to-end ingestion times were 193 seconds for PDFs, 116 seconds for audio, and 203 seconds for video.
• Query handling latency was around 2 seconds, with API response latency around 5 seconds, supporting up to 200 concurrent queries.
• There was a ~40% reduction in time to locate documents and a ~35% increase in search relevance in initial deployments.
• Automatic tagging saved ~50 hours per month in metadata curation, and manual inspection effort reduced by ~45%.

Figure 2: Vectorization Pipeline