slow responses... COPY and RENAME graphs

[cgokey]

I'm running RDF4j 5.1.x in AWS, running in docker as a ECS container service. The file system is using EBS. Config looks like this:
[] a config:Repository ;
config:rep.id "kms" ;
rdfs:label "kms" ;
config:rep.impl [
config:rep.type "openrdf:SailRepository" ;
config:sail.impl [
config:sail.type "openrdf:NativeStore" ;
config:native.forceSync true ;
config:sail.memory "false" ;
config:sail.reindex "true" ;
config:sail.writeThrough "true" ;
]
].
'

We've got 130+ graph, each with about 180K triples in them.

Using the RDF4J RESTful API, we have an operation that does
MOVE [published graph] TO [past_published_graph]
and as part of the same operation
COPY [draft]
TO [publish]

Locally this operation takes about 5 seconds to complete. But in AWS we are seeing 30+ seconds, enough for our API Gateway to timeout.

I'm surprised there such a difference between local and AWS, the server is under no load. I'm wondering if this might be latency since it using a EBS volume.

I've given the server 6G of memory, so it should have plenty for the data being stored.

Curious if there are any red flags for those who have worked with rdf4j in this environment, is this normal? Any thoughts on making this faster. We'll probably just call this operation asynchronous, but also seeing slower responses in other areas as well, so I thought I'd ask.

Thanks and appreciate any thoughts.
Chris

[hmottestad]

NativeStore isn't very fast in general. Have you tried with the LMDB Store?

[kenwenzel]

The default indexes are spoc and posc as can be seen here:
https://github.com/eclipse-rdf4j/rdf4j/blob/b33d91485502d2f5266916c0581960e41b8f28b5/core/sail/nativerdf/src/main/java/org/eclipse/rdf4j/sail/nativerdf/TripleStore.java#L62C49-L62C58

If you always query with a specific graph then you can change them to cspo and cpos or combine them with the existing ones:
spoc,posc,cspo,cpos

[cgokey]

Thanks, is it possible to request a query plan of the index order it uses for a given query?

[cgokey]

Ok that did the trick, thank you!
config:native.tripleIndexes "cspo,cpos,spoc,posc" ;

Question, if I want to test out lmbdstore in the same way, do I just use:
config:lmdb.tripleIndexes "cspo,cpos,spoc,posc" ;

[cgokey]

NativeStore isn't very fast in general. Have you tried with the LMDB Store?

I was hesitant mostly because last I checked it was labeled as experimental.

[kenwenzel]

Yes, that should work. For your specific case it could happen that the LmdbStore is slower as it uses garrbage collection for values if something is deleted.
It is marked experimental as the value encoding might change in future versions to support inlining of scalars and RDF-star triples (RDF 1.2).