Search optimization

[kabilov]

Hi!

Maybe I'm not quite right, but it seemed that loading the database into memory each time a new sample is analyzed is time-consuming. For example, in my case, loading GTDB into memory takes 30 minutes (with --load-whole-db) and if there are 10 samples, then 5 hours of time are lost. Maybe, following the example of the STAR (https://github.com/alexdobin/STAR) , separate commands for loading and unloading the database into and from memory should be introduced.
That is, the database is loaded once, all samples are analyzed, and then the memory is freed.

Best wishes,
Marsel

[shenwei356]

Did you try not using --load-whole-db? It won't take extra time to load the database, while it might be slow in cluster environments where the database is located in network-attached-storage.

[kabilov]

Time of analysis the same data (204 Mb):
3.5 h with --load-whole-db (0.5 h = loading database, 3 h = analysis)
6 h without --load-whole-db