Does skrl support transformer ?

[giangdao1402]

Hi @Toni-SM
Currently, i am developing a project using reinforcement learning for mobile robot navigation (I am using IssacLab)
My question is that does skrl currently support transformer ?
Thank you !

[Toni-SM]

Hi @giangdao1402

Currently yaml configuration integrated Isaac Lab for skrl doesn't support it.
And, even when there is not a transformer example in skrl' docs, you can use skrl's standalone script to implement it and train/evaluate on Isaac Lab as shown in skrl docs examples for NVIDIA Isaac Lab.

Also, please see #276 (reply in thread)

[giangdao1402]

Thank for your suggestion, i will try it

[giangdao1402]

Hi @Toni-SM
i am testing the GRU with PPO_RNN using the implementation of torch_gymnasium_pendulumnovel_ppo_gru.py for my task but i encountered a problem with tensor shape
[DEBUG] Action input 0: tensor([0.6335, 0.1261], device='cuda:0') [DEBUG] Action input 0: tensor([ 0.0000, -0.5029], device='cuda:0') 1%|█▋ | 1023/100000 [00:51<1:23:02, 19.86it/s] Error executing job with overrides: [] Traceback (most recent call last): File "/home/trash/IsaacLab/source/isaaclab_tasks/isaaclab_tasks/utils/hydra.py", line 101, in hydra_main func(env_cfg, agent_cfg, *args, **kwargs) File "/home/trash/VAE_navigation/scripts/skrl/train_PPO_GRU.py", line 258, in main trainner.train() File "/home/trash/miniconda3/envs/env_isaaclab/lib/python3.10/site-packages/skrl/trainers/torch/sequential.py", line 86, in train self.single_agent_train() File "/home/trash/miniconda3/envs/env_isaaclab/lib/python3.10/site-packages/skrl/trainers/torch/base.py", line 222, in single_agent_train self.agents.post_interaction(timestep=timestep, timesteps=self.timesteps) File "/home/trash/miniconda3/envs/env_isaaclab/lib/python3.10/site-packages/skrl/agents/torch/ppo/ppo_rnn.py", line 427, in post_interaction self._update(timestep, timesteps) File "/home/trash/miniconda3/envs/env_isaaclab/lib/python3.10/site-packages/skrl/agents/torch/ppo/ppo_rnn.py", line 575, in _update _, next_log_prob, _ = self.policy.act( File "/home/trash/miniconda3/envs/env_isaaclab/lib/python3.10/site-packages/skrl/models/torch/gaussian.py", line 129, in act mean_actions, log_std, outputs = self.compute(inputs, role) File "/home/trash/VAE_navigation/scripts/skrl/network_PPO_GRU.py", line 56, in compute rnn_input = states.view(-1, self.sequence_length, states.shape[-1]) # (N, L, Hin): N=batch_size, L=sequence_length RuntimeError: shape '[-1, 128, 135]' is invalid for input of size 421875
i think the error caused by the fixed sequence_length of GRU, but since im not quite familar with deep learning or GRU, i can not fix the error, can you give me some suggestions ?