📋 Content details

The content of the tutorial is planned as below:

• Introduction: Graph representation learning has emerged as a vital area of research, enabling the analysis of complex data represented as graphs [Zhang et al., 2020]. However, existing methods often struggle with noisy/incomplete structures, the loss of critical graph structures in the representation learning space, and failing to uncover the underlying structural factors. These challenges lead to distortion in graph representation learning, i.e., loss of information in the graph's intrinsic complex structure and topological properties, which inevitably degrades model performance. This tutorial will introduce research on low-distortion graph representation learning, encompassing information-theoretic graph representation learning, geometry-guided graph representation learning, and invariance-guided graph representation learning. The goal is to introduce state-of-the-art methods that can be utilized to minimize distortion while enhancing the performance of graph representation learning methods.
• Information-theoretic graph representation learning : Information theory [Kullback, 1997] provides a theoretical framework for efficiently learning, processing, and transmitting information, as well as representing it effectively [Principe, 2010], which aligns to some extent with the objectives of representation learning. Following the immense success of information-theoretic methods on image,language, audio, etc., very recently, there has been increasing interest in applying information-theoretic methods to the structured graph data. Analyzing and extracting the information from the complex node features and irregular structure is fundamental in graph learning, and important efforts have been invested in developing models to obtain low-distortion representations. By measuring the uncertainty, entropy facilitates efficient feature selection and prioritizes informative structures for complex graph data. Representative methods include REM [Liu et al., 2019], SEP [Wu et al., 2022a], etc. By quantifying the dependence and discrepancies between distributions, mutual information and divergence facilitate retaining the critical dependencies, and constraints enforce distributional realism. Representative methods include InfoGraph [Sun et al., 2020], GraphVAE [Simonovsky and Komodakis, 2018], etc. By balancing between information compression and retention, information theoretic principles such as the Information Bottleneck facilitate task-oriented lowdistortion representation learning. Representative methods include GIB [Wu et al., 2020], VIB-GSL [Sun et al., 2022], etc. By grounding graph representation learning in information theory, we achieve rigorous control over the distortioncompression trade-off, unlocking scalable and interpretable models for complex relational data.
• Geometry-guided graph representation learning : Graph Geometry, or Network Geometry, has raised as a burgeoning research focus within the graph learning community, providing novel perspectives for studying the topological structure and properties of graphs. Traditional machine learning methods typically learn data representations in Euclidean spaces, whereas the non-Euclidean geometric structure of graphs inevitably leads to topological distortion in their embeddings/representations. From the perspective of graph geometry, graph structures should be regarded as discretizations of continuous manifolds in non-Euclidean geometry. Riemannian geometry, in particular, provides an elegant and rigorous mathematical framework for advancing research in graph machine learning. Riemannian geometric space is a non-Euclidean geometric structure, which includes a spherical space with positive curvature, a Euclidean space with zero curvature and a hyperbolic space with negative curvature. Graph geometry has developed a lot of related work in recent years due to its ability to provide low-distortion priors and inductive biases for graph representation learning. It can be summarized into two types: Continuous space-based Riemannian graph learning and discrete curvature-based Euclidean graph learning. Riemannian geometric graph learning is to extend the graph learning from Euclidean space to the Riemannian space, to preserving the topological properties of the graph as much as possible to achieve the lowdistortion graph representations. Representative methods include HNN [Ganea et al., 2018], HGCN [Chami et al., 2019], Mixed-Curvature [Gu et al., 2019], κ-GCN [Bachmann et al., 2020], ACE-HGNN [Fu et al., 2021], etc. The discrete curvature Graph learning is to use curvature defined on discrete structures (e.g., Ricci curvature) as an inductive bias of traditional graph learning to improve the model to capture special structures or to explore and breakthrough the performance bottleneck of GNNs. Representative methods include CurvGN [Ye et al., 2019], CurvGAN [Li et al., 2022d], SDRF [Topping et al., 2022], CurvGIB [Fu et al., 2025], etc.
• Invariance-guided graph representation learning : Backed by causality theory, invariant graph learning aims to separate task-invariant and task-variant parts of the original graph. It assumes that the information of each graph instance includes two parts, i.e., an invariant part whose relationship with the task label is stable across different environments, and a variant part whose relationship with the label can change across different environments. By exploiting task-invariant patterns in representation learning while disregarding the variant spurious correlations, invariant graph learning can provably reduce distortion and make accurate predictions, especially under distribution shifts. Along this line, there are mainly two types of graph invariant learning methods: invariance optimization and explicit representation alignment. For invariance optimization, the goal is to optimize the representation learning model with invariance regularizers. Representative examples including GIL [Li et al., 2022c], DIR [Wu et al., 2022c], EERM [Wu et al., 2022b], FLOOD [Liu et al., 2023], DIDA [Zhang et al., 2022], etc. For explicit representation alignment, the key idea is to explicitly align the graph representations among multiple environments (or domains). Typical examples including SR-GNN [Zhu et al., 2021], StableGL [Zhang et al., 2023]. A more challenging scenario is when the downstream tasks are unknown, e.g., in the self-supervised training or pre-training paradigm. In these cases, the invariance principle is instantiated as de-correlating different dimensions or latent factors of representations. Typical examples including OOD-GNN [Li et al., 2022a], IDGCL [Li et al., 2022b], and OOD-GCL [Li et al., 2024].
• Advanced Directions: It is valuable to discuss the open issues and future directions:
  
  • More advanced theories. Though three categories of methods for low-distortion GRL have been discussed, existing theoretical analyses provide different angles for understanding distortion. Therefore, more advanced and potentially unified theories are still required to guarantee the effectiveness of low-distortion GRL.

  • Low-distortion in graph large language models (GraphLLM) and graph foundation models (GFM). Inspired by the success of LLMs in other domains, GraphLLM and GFM have received ever-increasing attention in the past two years. Existing low-distortion GRL mostly focuses on traditional models such as GNNs. How to combine the strengths of low-distortion GRL and these more advanced GRL models remains to be further explored.

  • Application in Graph for Science. With the fast development of interdisciplinary research, GRL has been applied to various scientific fields, such as bio-informatics, physical simulation, material designs, etc. How to apply lowdistortion GRL into these broad applications can further increase the application value and benefit broad audience in the scientific community.

References