in-graph and between-graph clustering

[StephDocTUM]

Dear all,

I want to do an in-graph and between graph clustering approach. Are there any methods already implemented especially for between-graph clustering algorithms?

Best,
Steph

[rusty1s]

Do you mean clustering of graphs rather than nodes inside a single graph? One could also use "Deep Graph Infomax" for this which is also integrated into PyG. Otherwise, please let me know of some references in which we can look into.

[StephDocTUM]

Yes, clustering of graphs. E.g. I have 1000 graphs and want to cluster the graphs into different clusters. So clustering a set of graphs:
e.g. https://web.iitd.ac.in/~bspanda/graphclustering.pdf

[rusty1s]

Yes, we indeed have not an example for this (only node-level clustering), but one should be able to customize the Deep Graph Infomax appraoch for this. Let me know if you are interested in working on this with us!

[StephDocTUM]

Yes, I would love to. Would write me an email to get started?

Best,
Stephan

[rusty1s]

No reason to go private for this. We can either discuss the roadmap here or in a separate issue. Let me know if you need any pointers on how to get started.

[StephDocTUM]

Yes sure, lets discuss the roadmap here. What do you think is a good way to start?

[rusty1s]

Fixed in #4402.

[StephDocTUM]

yes and When I only call DeepGraphInfomax.loss, where is my final summary vector of the positive graph updated, which I use in the end for clustering?
In DeepGraphInfomax the updating takes place in the forward method (I used as encoder Node2vec)?

[rusty1s]

I am not sure I understand. IMO, you can directly use the summary output of model.forward to perform clustering, right?

[StephDocTUM]

Yes.
One question for the summary function = readout function in DGI. I have to specify this function to be able to run the foward() function in DGI.
Where is this summary function defined and how does it look like?

[rusty1s]

The summary function is defined here: https://github.com/pyg-team/pytorch_geometric/blob/master/examples/infomax_transductive.py#L33

[StephDocTUM]

What about:

def corruption(x, edge_index):
    return x[torch.randperm(x.size(0))], edge_index

def readout(z, *args, **kwargs):
    return torch.sigmoid(z.mean(dim=0))

def train_one_epoch(model,data):
    data = data.to(DEVICE)
    x = data.x.to(DEVICE)
    edge_index = data.edge_index.to(DEVICE)
    model.train() #wo wird das ausgeführt?
    optimizer.zero_grad()
    pos_z, neg_z, summary = model(x.float(), edge_index) ###where does model go with data.x,..
    loss = model.loss(pos_z, neg_z, summary)
    loss.backward()
    optimizer.step()
    return loss.item()



## SETTING UP THE MODEL
feature_dim = data.x.shape[1]
model = DeepGraphInfomax(
        hidden_channels=EMB_DIM,
        encoder= Encoder(feature_dim,EMB_DIM),
        summary=readout,
        corruption=corruption).to(DEVICE)

# Embedding space before training the model
model.eval()
x = data.x.to(DEVICE)
edge_index = data.edge_index.to(DEVICE)
untrained_out = model(x.float(), edge_index)

# Training
optimizer = torch.optim.Adam(model.parameters(), lr=LR)

for epoch in range(EPOCH):

    loss = train_one_epoch(model,data)
    print('Epoch: {:03d}, Loss: {:.4f}'.format(epoch, loss))

# Embedding space after training
model.eval()
trained_out = model(x.float(), edge_index)

then

trained_out[2] should give me the vector of the embedding of my entire graph right?

[rusty1s]

Yes, that is correct.

[StephDocTUM]

So, this can be used for in-between graph clustering right?

[rusty1s]

I think so. You can use examples from the training set as corrupted examples.