GATv2Conv

class dgl.nn.pytorch.conv.GATv2Conv(in_feats, out_feats, num_heads, feat_drop=0.0, attn_drop=0.0, negative_slope=0.2, residual=False, activation=None, allow_zero_in_degree=False, bias=True, share_weights=False)[source]

Bases: Module

GATv2 来自 图注意力网络有多专注？

\[h_i^{(l+1)} = \sum_{j\in \mathcal{N}(i)} \alpha_{ij}^{(l)} W^{(l)}_{right} h_j^{(l)}\]

其中 \(\alpha_{ij}\) 是节点 \(i\) 和节点 \(j\) 之间的注意力分数：

\[ \begin{align}\begin{aligned}\alpha_{ij}^{(l)} &= \mathrm{softmax_i} (e_{ij}^{(l)})\\e_{ij}^{(l)} &= {\vec{a}^T}^{(l)}\mathrm{LeakyReLU}\left( W^{(l)}_{left} h_{i} + W^{(l)}_{right} h_{j}\right)\end{aligned}\end{align} \]
Parameters:

  • in_feats (int, 或 pair of ints) – 输入特征大小；即，\(h_i^{(l)}\) 的维度数。 如果该层要应用于单向二分图，in_feats 指定源节点和目标节点上的输入特征大小。 如果给定一个标量，源节点和目标节点的特征大小将取相同的值。

  • out_feats (int) – Output feature size; i.e, the number of dimensions of \(h_i^{(l+1)}\).

  • num_heads (int) – Number of heads in Multi-Head Attention.

  • feat_drop (float, optional) – Dropout rate on feature. Defaults: 0.

  • attn_drop (float, optional) – Dropout rate on attention weight. Defaults: 0.

  • negative_slope (float, optional) – LeakyReLU angle of negative slope. Defaults: 0.2.

  • residual (bool, optional) – If True, use residual connection. Defaults: False.

  • activation (callable activation function/layer or None, optional.) – If not None, applies an activation function to the updated node features. Default: None.

  • allow_zero_in_degree (bool, optional) – If there are 0-in-degree nodes in the graph, output for those nodes will be invalid since no message will be passed to those nodes. This is harmful for some applications causing silent performance regression. This module will raise a DGLError if it detects 0-in-degree nodes in input graph. By setting True, it will suppress the check and let the users handle it by themselves. Defaults: False.

  • bias (bool, 可选) – 如果设置为 False，该层将不会学习 一个加性偏差。（默认值：True）

  • share_weights (bool, 可选) – 如果设置为 True，则上述公式中的 \(W_{left}\) 和 \(W_{right}\) 的相同矩阵将应用于每条边的源节点和目标节点。 (默认: False)

注意

零入度节点将导致无效的输出值。这是因为没有消息会传递到这些节点，聚合函数将在空输入上应用。避免这种情况的常见做法是如果图是同质的，则为每个节点添加自环，这可以通过以下方式实现：

>>> g = ... # a DGLGraph
>>> g = dgl.add_self_loop(g)

Calling add_self_loop will not work for some graphs, for example, heterogeneous graph since the edge type can not be decided for self_loop edges. Set allow_zero_in_degree to True for those cases to unblock the code and handle zero-in-degree nodes manually. A common practise to handle this is to filter out the nodes with zero-in-degree when use after conv.

示例

>>> import dgl
>>> import numpy as np
>>> import torch as th
>>> from dgl.nn import GATv2Conv

>>> # Case 1: Homogeneous graph
>>> g = dgl.graph(([0,1,2,3,2,5], [1,2,3,4,0,3]))
>>> g = dgl.add_self_loop(g)
>>> feat = th.ones(6, 10)
>>> gatv2conv = GATv2Conv(10, 2, num_heads=3)
>>> res = gatv2conv(g, feat)
>>> res
tensor([[[ 1.9599,  1.0239],
        [ 3.2015, -0.5512],
        [ 2.3700, -2.2182]],
        [[ 1.9599,  1.0239],
        [ 3.2015, -0.5512],
        [ 2.3700, -2.2182]],
        [[ 1.9599,  1.0239],
        [ 3.2015, -0.5512],
        [ 2.3700, -2.2182]],
        [[ 1.9599,  1.0239],
        [ 3.2015, -0.5512],
        [ 2.3700, -2.2182]],
        [[ 1.9599,  1.0239],
        [ 3.2015, -0.5512],
        [ 2.3700, -2.2182]],
        [[ 1.9599,  1.0239],
        [ 3.2015, -0.5512],
        [ 2.3700, -2.2182]]], grad_fn=<GSpMMBackward>)

>>> # Case 2: Unidirectional bipartite graph
>>> u = [0, 1, 0, 0, 1]
>>> v = [0, 1, 2, 3, 2]
>>> g = dgl.heterograph({('A', 'r', 'B'): (u, v)})
>>> u_feat = th.tensor(np.random.rand(2, 5).astype(np.float32))
>>> v_feat = th.tensor(np.random.rand(4, 10).astype(np.float32))
>>> gatv2conv = GATv2Conv((5,10), 2, 3)
>>> res = gatv2conv(g, (u_feat, v_feat))
>>> res
tensor([[[-0.0935, -0.4273],
        [-1.1850,  0.1123],
        [-0.2002,  0.1155]],
        [[ 0.1908, -1.2095],
        [-0.0129,  0.6408],
        [-0.8135,  0.1157]],
        [[ 0.0596, -0.8487],
        [-0.5421,  0.4022],
        [-0.4805,  0.1156]],
        [[-0.0935, -0.4273],
        [-1.1850,  0.1123],
        [-0.2002,  0.1155]]], grad_fn=<GSpMMBackward>)
forward(graph, feat, get_attention=False)[source]

Description

计算图注意力网络层。

param graph:

图表。

type graph:

DGLGraph

param feat:

If a torch.Tensor is given, the input feature of shape \((N, D_{in})\) where \(D_{in}\) is size of input feature, \(N\) is the number of nodes. If a pair of torch.Tensor is given, the pair must contain two tensors of shape \((N_{in}, D_{in_{src}})\) and \((N_{out}, D_{in_{dst}})\).

type feat:

torch.Tensor 或一对 torch.Tensor

param get_attention:

是否返回注意力值。默认为False。

type get_attention:

布尔值，可选

returns:

  • torch.Tensor – 输出特征的形状为 \((N, H, D_{out})\)，其中 \(H\) 是头数，\(D_{out}\) 是输出特征的大小。

  • torch.Tensor, 可选 – 形状为 \((E, H, 1)\) 的注意力值，其中 \(E\) 是边的数量。仅当 get_attention 为 True 时返回。

raises DGLError:

If there are 0-in-degree nodes in the input graph, it will raise DGLError since no message will be passed to those nodes. This will cause invalid output. The error can be ignored by setting allow_zero_in_degree parameter to True.

reset_parameters()[source]

Description

重新初始化可学习的参数。

注意

fc权重\(W^{(l)}\)使用Glorot均匀初始化进行初始化。 注意力权重使用xavier初始化方法。