Beginner questions about environments training/evaluation

[LamannaLeonardo]

I am considering switching to this library for training RL agents, I am interested in recurrent model-free/based algorithms. I'd like to ask a few questions:

1. Does this library allows for parallel training on a training set of environments split into batches ? For example, given a training set of 20 environments, train the agent using 4 batches of 5 parallel environments (where the parallel environments are instantiated with different parameters).
2. Is it possible to evaluate the agent on a validation set of environments (and with possibly multiple episodes per validation environment), rather than on a single environment?
3. Does DI-engine allows for (possibly multiple) GPUs training and does it come with integrated hyperparameters tuning optimization utilities?

Thank you

[LamannaLeonardo]

To answer my own questions: 1 and 2 are possible as of the DQN example and n_evaluator_episode configuration parameter. Multi-GPUs training is apparently supported in the current documentation.

[PaParaZz1]

Thanks for your interest about DI-engine.

1. This setting is similar to our so-called multi-task training, you can refer to the detailed demo in our LightZero repo.
2. The second problem can be resolved by adjusting the set n_evaluator_episode argument.
3. The doc you found is about PyTorch DistributedDataParallel (DDP) multi-GPU training, if you need more complicated training backend like deepspeed, feel free to open a new issue—we’d be happy to assist and prioritize its implementation.

[LamannaLeonardo]

Thanks for replying @PaParaZz1. LightZero looks also very interesting! I will check it out