[Paper Review] Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks

Yujun Yan, Milad Hashemi+3 more|arXiv (Cornell University)|Feb 12, 2021

Advanced Graph Neural NetworksComputer Science43 references25 citations

TL;DR

The paper jointly explains oversmoothing and heterophily in GCNs using node-level metrics, and proposes GGCN with structure- and feature-based edge corrections to address both issues.

ABSTRACT

In node classification tasks, graph convolutional neural networks (GCNs) have demonstrated competitive performance over traditional methods on diverse graph data. However, it is known that the performance of GCNs degrades with increasing number of layers (oversmoothing problem) and recent studies have also shown that GCNs may perform worse in heterophilous graphs, where neighboring nodes tend to belong to different classes (heterophily problem). These two problems are usually viewed as unrelated, and thus are studied independently, often at the graph filter level from a spectral perspective. We are the first to take a unified perspective to jointly explain the oversmoothing and heterophily problems at the node level. Specifically, we profile the nodes via two quantitative metrics: the relative degree of a node (compared to its neighbors) and the node-level heterophily. Our theory shows that the interplay of these two profiling metrics defines three cases of node behaviors, which explain the oversmoothing and heterophily problems jointly and can predict the performance of GCNs. Based on insights from our theory, we show theoretically and empirically the effectiveness of two strategies: structure-based edge correction, which learns corrected edge weights from structural properties (i.e., degrees), and feature-based edge correction, which learns signed edge weights from node features. Compared to other approaches, which tend to handle well either heterophily or oversmoothing, we show that {our model, GGCN}, which incorporates the two strategies performs well in both problems.

Motivation & Objective

• Introduce theoretically-grounded node-level metrics to profile nodes across GCN layers (relative degree and node-level heterophily).
• Provide a joint explanation for oversmoothing and heterophily problems based on node behaviors.
• Develop and validate methods that correct edges structurally and via node features to mitigate both issues.

Proposed method

• Define node-level homophily h_i and relative degree overline{r_i} to characterize node behaviors across layers.
• Derive theoretical conditions (Theorems 3.1–3.3) linking node metrics to movements in node representations and misclassification.
• Propose structure-based edge correction by learning layer-specific edge scalars tau_{ij}^l to modulate neighbor influence.
• Propose feature-based edge correction by learning signed edge weights using cosine similarity between node representations to create positive/negative message paths.
• Incorporate decaying aggregation to slow representation convergence and improve stability.
• Present GGCN as a unified model combining the two edge-correction mechanisms and decaying aggregation.

Experimental results

Research questions

• RQ1 How do node-level metrics (relative degree and node-level homophily) explain oversmoothing and heterophily in GCNs?
• RQ2 Can edge-correction strategies based on structure and features mitigate both oversmoothing and heterophily, and how do they perform across datasets with varying homophily levels?
• RQ3 Do signed edges and layer-wise corrections improve robustness of GCNs to oversmoothing and heterophily compared to existing models?

Key findings

• The interplay of relative degree and node-level homophily defines three node behaviors across layers, explaining oversmoothing and heterophily jointly.
• Signed edge weights can help alleviate both problems under certain error-rate and degree conditions.
• Structure-based edge correction improves edge weights using degree-related properties to prevent oversmoothing in high-homophily graphs.
• Feature-based edge correction uses cosine-based signs to differentiate positive and negative neighbor influences, enhancing performance in heterophilous graphs.
• The proposed GGCN, combining both edge-correction strategies and decaying aggregation, achieves strong performance on high-heterophily datasets and competitive results on homophilous datasets.