Can knowledge graph embeddings be initialized from specific value?

[songsong0425]

Dear PyG community,

Greetings, thank you for your efforts on the package.
I have a general question about initializing knowledge graph embedding (KGE) models.
To my knowledge, most of the KGE models have used randomly initialized vectors for the node features.
Can the specific feature vectors be used for some nodes in KG?
For example, entity A use a 256-dimensional feature vector, otherwise uses a dimension-matched randomly initialized vector.
If it is impossible, can you explain why?

Thank you for reading this question.

[rusty1s]

I think you have two options:

1. You initialize the feature vector with a set of pre-defined features, and refine their features as training progresses
2. If you have additional node features, you concatenate/replace them to the embedding layer. This can be as easy as doing

x = torch.cat([data.x, self.emb.weight)], dim=-1)

[songsong0425]

Thank you for your kind reply! But I'm wondering when the concatenated x has a different dimension of vectors per node.
For example, nodes 0 ~ 4 have additional node features with 32 dimensions, and nodes 5 ~ 9 don't have features.
When the emb.weight has 128 dimensions and concat it to the nodes, nodes will have different dimensional vectors (i.e., 160 and 128, respectively).
Although I found a similar discussion in #7741 , should I match the dimension for all nodes? (I think I should do it so..)

[rusty1s]

You can use summation instead by first transforming features into the same dimensionality, e.g.:

lin = Linear(...)

x_feat = lin(x_feat)
emb[nodes_that_have_features] += x_feat

[songsong0425]

It looks like a good idea! Thank you for your suggestion.