Number of threads for CPU training

[VarIr]

We should document how to set the number of threads for training on CPUs (in case anyone would like to do that).
Basically, it's export OMP_NUM_THREADS=8 for intra-op parallelism.

Alternatively, this may be set programatically with torch.set_num_threads() and set_num_interop_threads().

[jsoghigian]

Hi, I'm looking to train a dataset to test DeepNOG with. Could you provide a bit more information on how to do this? I'm working on a cluster environment if that helps at all. Do I need to make any changes to deepnog train relative to the docs once I set OMP_NUM_THREADS?

[VarIr]

Last time I tried this, setting this env variable was sufficient to control the number of CPU cores. In general, I'd suggest to train models on GPU rather than CPU for massively improved efficiency.

[jsoghigian]

I'm trying out GPU training, but our partitions with GPUs here are not properly detected as having GPUs by torch as having CUDA available. I'm investigating that on my end, but wanted to try training a dataset in the mean time :)

Thanks for the clarification!

[VarIr]

In this case you might want to try training a small model, that is, not on root or bacteria level, but something like tax 5794. This might finish in finite time also on CPUs :)