The Geometry and Topology of Graph Neural Networks

THEORY.md — an expository account of why message-passing GNNs fail, and of the sheaf-theoretic geometry that addresses over-smoothing and heterophily, built around worked examples small enough to check by hand.

Message-passing GNNs over-smooth, over-squash, and underperform on heterophilic graphs. There is a body of published work — spectral, combinatorial, geometric, and topological — explaining why. THEORY.md walks through that literature across five lenses and grounds the sheaf-theoretic part in explicit low-dimensional computations.

The worked examples are my own construction:

• A $2\times2$ sheaf Laplacian in which flipping one restriction map from $+1$ to $-1$ exchanges which eigenvector survives diffusion — turning "neighbours should agree" into "neighbours should disagree," which is the mechanism by which sheaf diffusion addresses over-smoothing and heterophily.
• An explicit $4\times4$ sheaf Laplacian for $d=2$ stalks with a $90°$ edge rotation, and its kernel — showing concretely that harmonic signals satisfy $x_v = R^{-1}x_u$, i.e. that restriction maps act as parallel transport and "agreement" means aligned up to a learned rotation.
• Holonomy around a triangle as the discrete-curvature test for whether a consistent global assignment exists.

Scope, stated plainly

THEORY.md is exposition of published results, not original research. The theory is due to Hansen & Gebhart (2020), Bodnar et al. (2022), Bronstein et al. (2021), Topping et al. (2022), and the other authors cited in its reference list. What is mine is the pedagogical framing and the worked examples above — chosen and computed to make the published claims concrete and checkable.

The accompanying implementation

Intro_to_GraphNeuralNetworks.ipynb implements the baseline that THEORY.md analyses: a two-layer Graph Convolutional Network built from scratch as transform → aggregate → activate, trained on the Cora citation network (2,708 papers, 1,433 features, 7 classes) to ~75% test accuracy. Each cell is annotated with what it does and where it comes from.

It is deliberately a plain GCN — the flat, "just average your neighbours" prior that THEORY.md identifies as the trivial sheaf, and whose failure modes motivate everything else in the document.

The code follows the live-coding demo in Petar Veličković's talk Intro to graph neural networks (ML Tech Talks), which is what introduced me to the subject. Built with Spektral (Daniele Grattarola); the model follows Kipf & Welling, Semi-Supervised Classification with Graph Convolutional Networks (ICLR 2017).

Running it

Requires Python 3.9+, Spektral 1.2.0, TensorFlow 2.x, NumPy, SciPy, NetworkX, scikit-learn. Cora downloads automatically via Spektral.

pip install spektral

Then open the notebook in Jupyter, JupyterLab, or Colab and run top to bottom.

Reading order

If you are here for the mathematics, read THEORY.md on its own — it is self-contained and does not require the notebook. If you want the baseline model in front of you first, run the notebook, then read THEORY.md §2–§4.

Author

Mohammed Sharukh — MSc Mathematics, IIT Palakkad (2022).

License

MIT.