[Paper Review] Leffingwell Odor Dataset

Benjamín Sánchez-Lengeling, Jennifer N. Wei+4 more|arXiv (Cornell University)|Oct 23, 2019

Olfactory and Sensory Function Studies39 references95 citations

TL;DR

The paper trains graph neural networks on a curated, expert-labeled QSOR dataset to predict odor descriptors from molecular graphs, establishing a learned odor space and showing transferability to related tasks.

ABSTRACT

<strong>NOTE: It's easier to download this dataset from pyrfume. Here's how:</strong> <pre><code># First install pyrfume in your Python environment. This can be done easily with pip. # pip install pyrfume import pyrfume molecules = pyrfume.load_data('leffingwell/molecules.csv', remote=True) behavior = pyrfume.load_data('leffingwell/behavior.csv', remote=True) # e.g. to count the number of molecules with each descriptor behavior.sum().sort_values(ascending=False).astype(int) </code></pre> Predicting properties of molecules is an area of growing research in machine learning, particularly as models for learning from graph-valued inputs improve in sophistication and robustness. A molecular property prediction problem that has received comparatively little attention during this surge in research activity is building Structure-Odor Relationships (SOR) models (as opposed to Quantitative Structure-Activity Relationships, a term from medicinal chemistry). This is a 70+ year-old problem straddling chemistry, physics, neuroscience, and machine learning. To spur development on the SOR problem, we curated and cleaned a dataset of 3523 molecules associated with expert-labeled odor descriptors from the <em>Leffingwell PMP 2001</em> database. We provide featurizations of all molecules in the dataset using bit-based and count-based fingerprints, Mordred molecular descriptors, and the embeddings from our trained GNN model (Sanchez-Lengeling et al., 2019). This dataset is comprised of two files: <strong>leffingwell_data.csv</strong>: this contains molecular structures, and what they smell like, along with train, test, and cross-validation splits. More detail on the file structure is found in leffingwell_readme.pdf. <strong>leffingwell_embeddings.npz</strong>: this contains several featurizations of the molecules in the dataset. <strong>leffingwell_readme.pdf</strong>: a more detailed description of the data and its provenance, including expected performance metrics. <strong>LICENSE</strong>: a copy of the CC-BY-NC license language. The dataset, and all associated features, is freely available for research use under the CC-BY-NC license. If you use the data in a publication, please cite: <pre>@article{sanchez2019machine, title={Machine learning for scent: Learning generalizable perceptual representations of small molecules}, author={Sanchez-Lengeling, Benjamin and Wei, Jennifer N and Lee, Brian K and Gerkin, Richard C and Aspuru-Guzik, Al{\'a}n and Wiltschko, Alexander B}, journal={arXiv preprint arXiv:1910.10685}, year={2019} }</pre>

Motivation & Objective

• Motivate QSOR as a challenging, long-standing problem spanning chemistry and neuroscience.
• Create a large expert-labeled odor dataset by canonicalizing descriptors from perfume databases.
• Demonstrate that graph neural networks can predict odor descriptors from molecular graphs more effectively than traditional baselines.
• Show that learned odor embeddings capture perceptual structure and support transfer learning to new odor descriptors.

Proposed method

• Represent molecules as graphs with atoms as nodes and bonds as edges.
• Train Graph Neural Networks to predict 138 odor descriptors simultaneously (multi-label classification).
• Compare GNNs to baselines (random forests and k-NN) using RDKit bit fingerprints, Morgan fingerprints, and Mordred features.
• Use penultimate GNN layer outputs as fixed dimensional odor embeddings for analysis of global and local structure.
• Evaluate with AUROC, precision, and F1, reporting bootstrap-based confidence intervals.
• Provide an appendix with hyperparameter tuning details and architectural variants (GCN vs MPNN).

Experimental results

Research questions

• RQ1 Can GNNs learn a generalizable odor representation from molecular graphs for multiple odor descriptors?
• RQ2 Do learned odor embeddings reflect perceptual relationships globally (clusters by odor groups) and locally (perceptually similar neighbors)?
• RQ3 Are GNN embeddings transferable to predicting unseen or newly defined odor descriptors?
• RQ4 Do odor embeddings transfer to related olfaction prediction tasks beyond the training dataset?
• RQ5 How does GNN-based QSOR performance compare to traditional feature-based baselines across multiple descriptors?

Key findings

• GNNs achieve higher mean AUROC (0.894) than baselines like Mordred RF (0.850) and Morgan-based RF (0.845).
• GNNs outperform bit-based (bFP) and count-based (cFP) fingerprints in AUROC across most descriptors.
• GNN embeddings organize odor space globally by perceptual similarity, clustering descriptors (e.g., musk, cabbage, lily, grape) in meaningful regions.
• Locally, KNN using GNN embeddings retrieves perceptually similar molecules better than KNN on fingerprints (AUROC 0.818 vs 0.782).
• Embeddings enable transfer learning to unseen descriptors, outperforming Morgan fingerprints and Mordred features in ablation tests.
• In the DREAM Olfaction Prediction Challenge context, GNN embeddings perform competitively with the state-of-the-art on mean Pearson’s r (0.55 vs 0.54).