GNN Layers#

GraphConvolutionalNetwork#

class supar.modules.gnn.GraphConvolutionalNetwork(n_model: int, n_layers: int = 1, selfloop: bool = True, dropout: float = 0.0, norm: bool = True)[source]#

Multiple GCN layers with layer normalization and residual connections, each executing the operator from the “Semi-supervised Classification with Graph Convolutional Networks” paper

\[\mathbf{X}^{\prime} = \mathbf{\hat{D}}^{-1/2} \mathbf{\hat{A}} \mathbf{\hat{D}}^{-1/2} \mathbf{X} \mathbf{\Theta},\]

where \(\mathbf{\hat{A}} = \mathbf{A} + \mathbf{I}\) denotes the adjacency matrix with inserted self-loops and \(\hat{D}_{ii} = \sum_{j=0} \hat{A}_{ij}\) its diagonal degree matrix.

Its node-wise formulation is given by:

\[\mathbf{x}^{\prime}_i = \mathbf{\Theta}^{\top} \sum_{j \in \mathcal{N}(v) \cup \{ i \}} \frac{e_{j,i}}{\sqrt{\hat{d}_j \hat{d}_i}} \mathbf{x}_j\]

with \(\hat{d}_i = 1 + \sum_{j \in \mathcal{N}(i)} e_{j,i}\), where \(e_{j,i}\) denotes the edge weight from source node j to target node i (default: 1.0)

Parameters:

  • n_model (int) – The size of node feature vectors.

  • n_layers (int) – The number of GCN layers. Default: 1.

  • selfloop (bool) – If True, adds self-loops to adjacent matrices. Default: True.

  • dropout (float) – The probability of feature vector elements to be zeroed. Default: 0.

  • norm (bool) – If True, adds a LayerNorm layer after each GCN layer. Default: True.

forward(x: Tensor, adj: Tensor, mask: BoolTensor) → Tensor[source]#
Parameters:

  • x (Tensor) – Node feature tensors of shape [batch_size, seq_len, n_model].

  • adj (Tensor) – Adjacent matrix of shape [batch_size, seq_len, seq_len].

  • mask (BoolTensor) – [batch_size, seq_len]. The mask for covering the unpadded tokens in each chart.

Returns:

Node feature tensors of shape [batch_size, seq_len, n_model].

Return type:

Tensor