Skip to yearly menu bar Skip to main content


Oral Poster

Exploitation of a Latent Mechanism in Graph Contrastive Learning: Representation Scattering

Dongxiao He · Lianze Shan · Jitao Zhao · Hengrui Zhang · Zhen Wang · Weixiong Zhang

[ ]
Fri 13 Dec 11 a.m. PST — 2 p.m. PST
 
Oral presentation: Oral Session 5B
Fri 13 Dec 10 a.m. PST — 11 a.m. PST

Abstract:

Graph Contrastive Learning (GCL) has attracted significant attention as it can reduce the heavy dependence of manual labels in training graph neural networks. Most existing GCL methods adopt one of the following frameworks: the InfoNCE-based framework, the DGI framework, or the BGRL framework. Unlike visual contrastive learning, the implementation of these three frameworks in GCL, particularly for node-level tasks, differs significantly. InfoNCE-based methods aim to achieve global uniformity by reducing the similarity between each pair of nodes. In contrast, DGI-like methods treat all nodes in the graph as a single class to distinguish them from their augmented counterparts. Despite their independent approaches, these frameworks achieve similar performances. Hence, we are motivated to investigate whether a common factor underlies the success of these seemingly disparate frameworks. We revisit these methods and discover that they collectively harness a key mechanism, representation scattering, that plays a crucial role in their success. However, existing GCL methods have not fully exploited this mechanism, ignoring its importance. We give a detailed proof in this paper. Based on this discovery, we propose a new contrastive learning method named SGRL. Specifically, SGRL introduces a representation scattering mechanism that explicitly and effectively scatters representations and guides model training. Furthermore, considering the linked nature of nodes, SGRL includes a topology-based constraint mechanism that adaptively integrates representation scattering with topological information. Extensive results from various downstream tasks on datasets of different scales have demonstrated the effectiveness and efficiency of SGRL.

Live content is unavailable. Log in and register to view live content