[Paper Review] Neural Message Passing for Quantum Chemistry

Justin Gilmer, Samuel S. Schoenholz+3 more|arXiv (Cornell University)|Apr 4, 2017

Computational Drug Discovery Methods2,978 citations

TL;DR

The paper reformulates graph-based neural models as Message Passing Neural Networks (MPNNs) and achieves state-of-the-art results on QM9, predicting DFT-level properties with chemical accuracy on most targets. It also introduces architectural variants to improve efficiency and performance.

ABSTRACT

Supervised learning on molecules has incredible potential to be useful in chemistry, drug discovery, and materials science. Luckily, several promising and closely related neural network models invariant to molecular symmetries have already been described in the literature. These models learn a message passing algorithm and aggregation procedure to compute a function of their entire input graph. At this point, the next step is to find a particularly effective variant of this general approach and apply it to chemical prediction benchmarks until we either solve them or reach the limits of the approach. In this paper, we reformulate existing models into a single common framework we call Message Passing Neural Networks (MPNNs) and explore additional novel variations within this framework. Using MPNNs we demonstrate state of the art results on an important molecular property prediction benchmark; these results are strong enough that we believe future work should focus on datasets with larger molecules or more accurate ground truth labels.

Motivation & Objective

• Demonstrate that a unified message passing framework can learn molecular properties directly from graph representations.
• Evaluate MPNNs on the QM9 dataset to benchmark performance against established baselines.
• Explore architectural variations (edge representations, readouts, towers) to maximize accuracy and efficiency.
• Investigate the importance of including spatial information and hydrogen treatment in molecular graphs.

Proposed method

• Define the Message Passing Neural Network (MPNN) framework with two phases: message passing and readout.
• Experiment with various message functions (edge-based, edge network, pair messages) and readout functions (GG-NN style and set2set).
• Incorporate virtual graph elements (virtual edges, master node) to enable long-range information flow.
• Use multiple towers (split hidden states into smaller copies) to increase capacity and speed without increasing computation time.
• Train one model per target on QM9, with hyperparameter search, and evaluate against chemical accuracy benchmarks.

Experimental results

Research questions

• RQ1 Can MPNNs learn molecular properties directly from graph representations with invariances to graph isomorphism?
• RQ2 How do different message, update, and readout choices affect accuracy on QM9 properties?
• RQ3 What is the impact of including spatial information and hydrogen explicitness on predictive performance?
• RQ4 Do architectural innovations (virtual edges, master node, set2set, towers) improve accuracy and efficiency for modest-sized molecules?

Key findings

• MPNNs achieve state-of-the-art results on all 13 QM9 targets.
• The best MPNN variant reaches chemical accuracy on 11 of the 13 targets (DFT-aligned ground truth).
• Models operating only on topology (no spatial input) can reach chemical accuracy on several targets when long-range interactions are captured.
• Using edge-network based messages and set2set readout with explicit hydrogens yields strong performance gains over baselines.
• A multi-tower approach provides speedups and improved generalization, outperforming a single-copy GG-NN in most targets.