Improving the Average Precision Score and F1 score using pytorch GAT

[amalislam675]

After the application of GAT using pytorch geometric framework, I am getting very low average precision score and F1 score. Is there any way to improve this score?

[rusty1s]

Can you share some more information here? This is generally hard to answer without :)

Do you see better performance when using other types of GNN layers, e.g., SAGEConv?

[rusty1s]

I suggest you try SAGEConv with multiple aggregations, e.g., aggr=['sum', 'mean', 'max]. Otherwise, I suggest to try GENConv as well.

[amalislam675]

@rusty1s , thankyou for your quick and great response on my question. Due to some limitation, I am not allowed to use GENConv, because here DeeperGCN is used. I have to use GNN model other than GCN. For GraphSAGE layer paper, there are two links: one is this link where we can use edge weight also https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.nn.models.GraphSAGE.html#torch_geometric.nn.models.GraphSAGE.
And the second one is in this link, https://pytorch-geometric.readthedocs.io/en/latest/generated/torch_geometric.nn.conv.SAGEConv.html. In SAGEConv layer, edge weight option is not not present. Can you suggest, which one should I follow? My multilabel classification task is protein function prediction. When I am making the model using graph sage, till three layers my F1 score become increase, after that, F1 score is start decreasing. Can you suggest any other GNN model through which I could use to make graph sage model model as a deeper gsage model as same as DeeperGCN. Or, can you suggest any other model except of GENConv with which F1 score for my problem could increase?

[rusty1s]

You can use GraphConv instead of SAGEConv if you want to use edge_weight.

[amalislam675]

@rusty1s , by following your suggestion, I am applying GraphConv from Pytorch geometric on my data. But I am facing this below mentioned error. Can you tell about it. On the same data I applied GatV2 from pytorch geometric, it code run sooth, but the performance was not better, so I am applying GraphConv.

Error

RuntimeError: CUDA out of memory. Tried to allocate 26.50 GiB (GPU 1; 23.65 GiB total capacity; 557.53 MiB already allocated; 20.47 GiB free; 572.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/xyz/pr/Project/WL_graphconv/src/method/main.py", line 162, in <module>
    train(model, optimizer, train_data)
  File "/xyz/pr/Project/WL_graphconv/src/method/main.py", line 54, in train
    trn_loss = train_epoch()
  File "/xyz/pr/Project/WL_graphconv/src/method/main.py", line 37, in train_epoch
    score = model(train_data["feature1"], train_data["edge_index1"], train_data["edge_weight1"], train_data["feature2"], train_data["edge_index2"], train_data["edge_weight2"], train_data["feature3"], train_data["edge_index3"], train_data["edge_weight3"])
  File "/xyz/.conda/envs/PY37_1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/xyz/pr/Project/WL_graphconv/src/method/model.py", line 19, in forward
    h1_1 = self.conv1_1(h1_1, ed_index1, ed_weight1)
  File "//xyz/.conda/envs/PY37_1/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/xyz/.conda/envs/PY37_1/lib/python3.7/site-packages/torch_geometric/nn/conv/graph_conv.py", line 80, in forward
    size=size)
  File "/xyz/.conda/envs/PY37_1/lib/python3.7/site-packages/torch_geometric/nn/conv/message_passing.py", line 430, in propagate
    size, kwargs)
  File "/xyz/.conda/envs/PY37_1/lib/python3.7/site-packages/torch_geometric/nn/conv/message_passing.py", line 301, in __collect__
    data = self.__lift__(data, edge_index, dim)
  File "/xyz/.conda/envs/PY37_1/lib/python3.7/site-packages/torch_geometric/nn/conv/message_passing.py", line 243, in __lift__
    f"Encountered a CUDA error. Please ensure that all "
ValueError: Encountered a CUDA error. Please ensure that all indices in 'edge_index' point to valid indices in the interval [0, 4705) in your node feature matrix and try again.

[akihironitta]

Hi @amalislam675, #6417 improved the error message, and so in a newer release, you should only see:

RuntimeError: CUDA out of memory. Tried to allocate 26.50 GiB (GPU 1; 23.65 GiB total capacity; 557.53 MiB already allocated; 20.47 GiB free; 572.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

which is basically telling you that your program is allocating more memory than your GPU can hold.