[Paper Review] Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models

Yin Fang, Xiaozhuan Liang+6 more|arXiv (Cornell University)|Jun 13, 2023

Machine Learning in Materials Science38 citations

TL;DR

Mol-Instructions presents a large-scale biomolecular instruction dataset for LLMs, covering molecule-, protein-, and biomolecular text-oriented tasks, and demonstrates improvements through instruction tuning on multiple models. It is publicly available for ongoing research with regular updates.

ABSTRACT

Large Language Models (LLMs), with their remarkable task-handling capabilities and innovative outputs, have catalyzed significant advancements across a spectrum of fields. However, their proficiency within specialized domains such as biomolecular studies remains limited. To address this challenge, we introduce Mol-Instructions, a comprehensive instruction dataset designed for the biomolecular domain. Mol-Instructions encompasses three key components: molecule-oriented instructions, protein-oriented instructions, and biomolecular text instructions. Each component aims to improve the understanding and prediction capabilities of LLMs concerning biomolecular features and behaviors. Through extensive instruction tuning experiments on LLMs, we demonstrate the effectiveness of Mol-Instructions in enhancing large models' performance in the intricate realm of biomolecular studies, thus fostering progress in the biomolecular research community. Mol-Instructions is publicly available for ongoing research and will undergo regular updates to enhance its applicability.

Motivation & Objective

• Motivate the creation of a dedicated biomolecular instruction dataset to fill a gap for LLMs.
• Construct Mol-Instructions with three core components: molecule-oriented, protein-oriented, and biomolecular text instructions.
• Demonstrate the effectiveness of instruction tuning on LLMs using Mol-Instructions across multiple baselines.
• Provide public access to the dataset and outline future enhancements to broaden applicability.

Proposed method

• Assemble over 2 million biomolecular instructions spanning three domains via mixed construction methods (self-instruct, template-based conversion, and human-crafted descriptions).
• Implement human–AI collaboration to generate diverse task descriptions using GPT-3.5-turbo with manual quality checks.
• Source data from standard biochemistry databases and PubMed; derive inputs/outputs, QA pairs, and design directives through data mining and AI-assisted generation.
• Convert biological data into textual formats using templates, including UniProtKB-based protein design annotations, to meet user-specified objectives.
• Apply stringent quality control: replace SMILES with SELFIES for molecules, curate UniProtKB entries, and cluster proteins with MMseqs at 90% similarity to reduce redundancy.
• Evaluate through instruction tuning on LLama-7B and other baselines across the three instruction domains, using training/validation/test splits.

Experimental results

Research questions

• RQ1 Does Mol-Instructions improve LLM performance on biomolecular understanding and generation tasks compared to baselines?
• RQ2 How do molecule-, protein-, and text-oriented instructions contribute to improvements across their respective tasks?
• RQ3 Can generated protein designs and molecular descriptions align with known functional or structural annotations?
• RQ4 What is the impact of dataset construction choices (self-instruct, templates, human-crafted descriptions) on model performance?

Key findings

• Mol-Instructions yields notable improvements in molecular understanding tasks across evaluated models and metrics compared to baselines.
• The data enables improved performance in molecular property prediction and generation tasks, with generated molecules showing higher similarity to reference structures.
• For protein-related tasks, tuned models demonstrate the ability to identify fundamental protein features and align de novo designs with UniProtKB annotations, suggesting functional relevance.
• Mol-Instructions enhances performance on biomolecular NLP tasks, including information extraction and Q&A in bioinformatics contexts.
• Compared to domain-specific smaller models, large models trained with Mol-Instructions still exhibit gaps in specialized generation, but show broader domain understanding gains.