[DISCUSSION] How to speed up interaction via anvio-interactive or anvio-refine on brower?

[shanexuuu]

Hi,

Thanks for developing such an amazing software! It is really convenient and user-friendly.

Recently, I am using anvio-8 on a linux server (1TB RAM, CPU AMD EPYC 9654, no GPU ) to mannually refine some eukaryotic bins (similar to what Tom did in here https://doi.org/10.1016/j.xgen.2022.100123). However, when I connect the server via ssh and open google-chrome to interact with anvio, it was really slow....It took like 5s to response just one click, which make my refinement take forever...

I was wondering is there any way to improve the interaction on brower? Is it related to my hardware of server? I am also considering to buy a new personal PC with GPU such as 4060/5060 for this purpose...

Many thanks and looking forward to hearing back!

The code I used:
anvi-refine -c CONTIGS.db \ # CONTIGS.db around 4GB
-p Merged_profile/PROFILE.db \ #PROFILE.db around 19 GB including 50 mapped samples
-C Refine
-b bin179 #bins around 10-70 Mbp long

[meren]

Hey @shanexuuu,

Unfortunately there are two things to consider here.

• Anvi'o interactive interface is not designed to handle more than 100,000 SVG objects at a given time. You can get an approximate idea about the number of SVG objects that will be displayed by multiplying the number of items in a display (the number of 'splits' in the case of anvi-refine with the number of layers (i.e., the number of samples in the profile-db). If the number exceeds 100,000 quite a lot, then it will start to get slow and memory-heavy, but there is no escape from that. There is no solution for this one :)
• Running the anvi'o interactive interface for large projects (such as refinement efforts) through SSH is going to be VERY VERY slow. There is no easy way to solve this since there are so many moving parts outside of anvi'o (the network connection, server response time (which will always be slower than personal computers for small and frequent requests), and SSH encryption/decryption for every single object). The solution is to download the data files on your computer, and do the refinement there. Then you can export the collection for refined bins, upload to the server, and import it into the original profile-db for downstream summary tasks.
• Running anvi-refine per bin for very large projects will also be extremely time consuming since initializing things will take a lot of time. One solution to that is to run anvi-split first on your collection, so every bin have their individual contigs-db and profile-db files which then can be used with anvi-refine. At the end of this one can export all collections, merge them, upload to the server, and import it into the original profile-db for downstream summary tasks.

I hope these help.

[shanexuuu]

Thanks for your quick response! It helps a lot, especially "anvi-split".

Another finding is if I use same Internet (such my university's local area network) for server and ssh, it would speed up a lot as well! Now I finally can run anvio interactive interface quickly.

Cheers,
Shane