Implementation Suggestion: HDT Walk Generation Algorithms

[cbobed]

Hi @janothan ,

I've been playing a little bit around, and I have seen that not all walk algorithms are implemented for HDT datasets. Is there any planning to include them? I might save time to help in the task.

Best,

[janothan]

Hi, @cbobed sorry for the late reply.
The HDT implementations should be included but I do not have much time currently.
Feel free to create a pull request if you already implemented something, any help is appreciated here 😊

Do you really need HDT for your work? There is currently only a basic walk implementation because HDT is quite slow. If you have much memory available, I do not recommend to use HDT.

[cbobed]

Hi, don't worry at all.

I don't exactly need it, but I helped with some implementations to calculate PageRank and some variations of centrality measures where having the graph in HDT helped a lot to scalability. Regarding the slowness you mention, when I had all the auxiliar indices created, I found it really fast for simple triples lookup regardless the direction of the query.

If I find some time in the following weeks, I'll try to contribute with some code (currently I have to meet a deadline) in this line. The good point of HDT was that it was threadsafe, and I could easily parallelize a lot of accesses; besides, in order to increase the scope of the approach it could be interesting (laptops with less memory, but with an SSD, for example).

Best,