speedup by multiprocessing with GPUs

[yudaiOfRiver]

Hi, I am stuck in implementing parallelization of solving problems.
I coded as follows with reference to the Starmap section on this page.

from pymoo.core.problem import Problem, ElementwiseProblem
from pymoo.core.problem import StarmapParallelization
import torch.multiprocessing as mp

class Myproblem(ElementwiseProblem):
.........

if name == 'main':
    n_proccess = 1000
    with mp.Pool(n_proccess) as pool:
        runner = StarmapParallelization(pool.starmap)
        params['elementwise_runner'] = runner
        problem = Myproblem(**params)

However, I got errors shown below...

OpenBLAS blas_thread_init: pthread_create failed for thread 54 of 56: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 1541886 current, 1541886 max

Does anyone know ways to achieve this speedup using GPUs, even if it is not through Starmap? By the way, I cannot use vectorized matrix operations for some reason.

[blankjul]

Does it work with fewer processes? How many CPUs does your machine have?

[yudaiOfRiver]

Thank you for your response.
Yes, about 100 processes work.

But I would like to make use of 5000 GPUs that my server has. And as far as I know, I could use it with torch.multiprocessing.
(By the way, the number of CPUs are about 30)

[blankjul]

In the evaluate function you have all the freedom to use the hardware the best way. Probably PyTorch or directly cuda code works best.