Edge classification with GCN or GAT

[lfangyu09]

Hi,

Thank you so much for providing this library. I want to do Edge Classification. I did not find an exact edge classification example in PyG. Therefore, I modified (one GCN example code) for edge classification.

class DeeperGCN(torch.nn.Module):
    def __init__(self, hidden_channels, num_layers):
        super().__init__()

        self.node_encoder = Linear(data.x.size(-1), hidden_channels)
        self.edge_encoder = Linear(data.edge_attr.size(-1), hidden_channels)

        self.layers = torch.nn.ModuleList()
        for i in range(1, num_layers + 1):
            conv = GENConv(hidden_channels, hidden_channels, aggr='softmax',
                           t=1.0, learn_t=True, num_layers=2, norm='layer')
            norm = LayerNorm(hidden_channels, elementwise_affine=True)
            act = ReLU(inplace=True)

            layer = DeepGCNLayer(conv, norm, act, block='res+', dropout=0.1,
                                 ckpt_grad=i % 3)
            self.layers.append(layer)

        self.lin = Linear(hidden_channels*3, data.y.size(-1))

    def forward(self, x, edge_index, edge_attr):
        x = self.node_encoder(x)
        edge_attr = self.edge_encoder(edge_attr)

        x = self.layers[0].conv(x, edge_index, edge_attr)

        for layer in self.layers[1:]:
            x = layer(x, edge_index, edge_attr)

        x = self.layers[0].act(self.layers[0].norm(x))
        x = F.dropout(x, p=0.1, training=self.training)
        
        x_src, x_dst = x[edge_index[0]], x[edge_index[1]]
        edge_feat = torch.cat([x_src, edge_attr, x_dst], dim=-1)
        out = self.lin(edge_feat)

        return out

I have three questions:

1. How to update edge features in each GCN layer?

for layer in self.layers[1:]:
            x = layer(x, edge_index, edge_attr)

Or which model could be used to update edge features?
2. I added these codes for edge classification. Is it correct?

x_src, x_dst = x[edge_index[0]], x[edge_index[1]]
edge_feat = torch.cat([x_src, edge_attr, x_dst], dim=-1)
out = self.lin(edge_feat)

3. Could I also add a residual connection in each layer? Is it good to improve accuracy?

for layer in self.layers[1:]:
            x = layer(x, edge_index, edge_attr)
            x += x

Thank you so much!

[rusty1s]

1. You can update the edge attributes as you described in point (2) after each GNN layer.
2. Yes.
3. Usually yes. GENConv applies this residual connection already, so in your case it would be redundant.

[lfangyu09]

Thank you so much for your prompt reply. May I ask four following questions:
1 I checked the codes of GENConv and the output only includes nodes. Therefore, if I want to also update the edge_attr in each layer, do I need to do the following for each layer?

for layer in self.layers[1:]:
  x = layer(x, edge_index, edge_attr)
  x_src, x_dst = x[edge_index[0]], x[edge_index[1]]
  edge_feat = torch.cat([x_src, edge_attr, x_dst], dim=-1)
  edge_attr = self.lin(edge_feat)

2 The codes of MetaLayer 's EdgeModel uses torch.cat([src, dst, edge_attr, u[batch]], 1), but here the code uses torch.cat([x_src, edge_attr, x_dst], dim=-1). Are the positions of dst and edge_attr in torch.cat important or not?

3 If I want to replace GENConv with GATConv in each GNN layer, how can I do that, especially I want to add more heads (e.g., heads=8 )? Such as:

self.convs = torch.nn.ModuleList()
for i in range(1, num_layers + 1):
    conv = GATConv(hidden_channels, hidden_channels, dropout=dropout)
    norm = BatchNorm1d(hidden_channels)
    act = LeakyReLU()
    self.convs.append(
        DeepGCNLayer(conv, norm, act, block='res+', dropout=dropout))

4 DeepGCNLayer provides several options, such as res+, res, dense. But I found example codes usually use res+. Is res+ better than res and dense?

Thank you so much!

[rusty1s]

1. Yes. I think it is easiest if you update edges outside of the GNN layer and then alternate between the two, such that GNN layers also always receive updated edge features.
2. No, the order is not important.
3. Not sure I understand. Can't you simply set heads=8?
4. I think that is something you may wanna tune on a specific dataset.