[Paper Review] Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond

Jonathan Godwin, Michael Schaarschmidt+6 more|arXiv (Cornell University)|Jun 15, 2021

Machine Learning in Materials Science8 citations

TL;DR

This paper introduces Noisy Nodes, a simple yet effective GNN regularisation technique that combats oversmoothing by corrupting input graph features with noise and adding a per-node denoising loss. The method enhances latent node diversity and manifold learning, achieving state-of-the-art performance on 3D molecular property prediction benchmarks—including 43% improvement on OC20 IS2RE direct—while improving results across OGB and GCN-based models.

ABSTRACT

In this paper we show that simple noise regularisation can be an effective way to address GNN oversmoothing. First we argue that regularisers addressing oversmoothing should both penalise node latent similarity and encourage meaningful node representations. From this observation we derive "Noisy Nodes", a simple technique in which we corrupt the input graph with noise, and add a noise correcting node-level loss. The diverse node level loss encourages latent node diversity, and the denoising objective encourages graph manifold learning. Our regulariser applies well-studied methods in simple, straightforward ways which allow even generic architectures to overcome oversmoothing and achieve state of the art results on quantum chemistry tasks, and improve results significantly on Open Graph Benchmark (OGB) datasets. Our results suggest Noisy Nodes can serve as a complementary building block in the GNN toolkit.

Motivation & Objective

• To address the problem of GNN oversmoothing, where node representations become indistinguishable after multiple message-passing layers.
• To develop a regularisation technique that encourages both node representation diversity and meaningful relational structure encoding.
• To demonstrate that simple, generic GNN architectures can achieve state-of-the-art performance when combined with Noisy Nodes.
• To provide a complementary, plug-and-play regularisation method applicable across diverse GNN architectures and graph learning tasks.

Proposed method

• Corrupt input node features with random noise during training to disrupt over-smoothed representations.
• Introduce a per-node denoising loss that encourages the model to reconstruct original features from noisy inputs.
• Train the GNN with a combined loss: primary prediction loss and auxiliary denoising loss to promote robust, diverse node representations.
• Apply the method to various GNN architectures, including Graph Network Simulator (GNS), MPNN, and GCN, across 3D molecular and non-spatial graph datasets.
• Use noise types such as random flipping of categorical features and feature dropout, with hyperparameter tuning for optimal balance between primary and auxiliary losses.
• Leverage the denoising objective to implicitly learn the underlying data manifold, improving generalisation and resistance to oversmoothing.

Experimental results

Research questions

• RQ1 Can a simple, noise-based regularisation technique effectively mitigate GNN oversmoothing across diverse architectures and datasets?
• RQ2 Does the combination of noise corruption and denoising loss improve node representation diversity and graph-level predictive performance?
• RQ3 Can Noisy Nodes enable shallow or otherwise underperforming GNNs—like the GNS architecture—to achieve state-of-the-art results on 3D molecular property prediction tasks?
• RQ4 How does Noisy Nodes compare to existing regularisation methods like DropEdge and DropNode in addressing oversmoothing?
• RQ5 Does the benefit of Noisy Nodes scale with model depth, and can it enable deeper GNNs to outperform shallower ones on tasks where depth was previously ineffective?

Key findings

• Noisy Nodes achieved a 43% improvement over prior work on the OC20 IS2RE direct molecular property prediction benchmark.
• The method improved MPNN performance on OGBG-PCQM4Mv1 to 28.1% ± 0.002 Mean Average Precision, surpassing baseline models.
• Applying Noisy Nodes to a 4-layer GCN on OGBN-Arxiv increased accuracy from 71.71% to 72.52%, demonstrating effectiveness even on simple GNNs.
• The technique enabled the GNS architecture—previously non-competitive—to achieve state-of-the-art results on three 3D molecular tasks, including QM9 and OC20.
• Validation curves show consistent performance gains with Noisy Nodes across training, indicating stable and effective regularisation.
• The method is effective across model depths, with diminishing returns at very deep architectures, suggesting a balance between depth and regularisation is key.