how to launch ARM instead of C2FARM

[softgearko]

Hello.
I tried to run launch with ARM as a method instead of C2FARM.

python launch.py method=ARM rlbench.task=take_lid_off_saucepan rlbench.demo_path=/home/softgear/stepjam_ARM/my_save_dir framework.gpu=0
I met warnings and errors as following. How can I launch ARM ?

launch.py:332: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name='config', config_path='conf')
/home/softgear/.local/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
  warnings.warn(msg, UserWarning)
/home/softgear/.local/lib/python3.8/site-packages/hydra/core/default_element.py:124: UserWarning: In 'method/ARM': Usage of deprecated keyword in package header '# @package _group_'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
  deprecation_warning(
/home/softgear/.local/lib/python3.8/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
  ret = run_job(
[2022-11-01 11:38:44,226][root][INFO] - 
method:
  name: ARM
  activation: lrelu
  q_conf: true
  alpha: 0.05
  alpha_lr: 0.0001
  alpha_auto_tune: false
  next_best_pose_critic_lr: 0.0025
  next_best_pose_actor_lr: 0.001
  next_best_pose_critic_weight_decay: 1.0e-05
  next_best_pose_actor_weight_decay: 1.0e-05
  crop_shape:
  - 16
  - 16
  next_best_pose_tau: 0.005
  next_best_pose_critic_grad_clip: 5
  next_best_pose_actor_grad_clip: 5
  qattention_grad_clip: 5
  qattention_tau: 0.005
  qattention_lr: 0.0005
  qattention_weight_decay: 1.0e-05
  qattention_lambda_qreg: 1.0e-07
  demo_augmentation: true
  demo_augmentation_every_n: 10
rlbench:
  task: take_lid_off_saucepan
  demos: 10
  demo_path: /home/softgear/stepjam_ARM/my_save_dir
  episode_length: 10
  cameras:
  - front
  camera_resolution:
  - 128
  - 128
  scene_bounds:
  - -0.3
  - -0.5
  - 0.6
  - 0.7
  - 0.5
  - 1.6
replay:
  batch_size: 128
  timesteps: 1
  prioritisation: true
  use_disk: false
  path: /tmp/arm/replay
framework:
  log_freq: 100
  save_freq: 100
  train_envs: 1
  eval_envs: 1
  replay_ratio: 128
  transitions_before_train: 200
  tensorboard_logging: true
  csv_logging: true
  training_iterations: 40000
  gpu: 0
  env_gpu: 0
  logdir: /tmp/arm_test/
  seeds: 1

[2022-11-01 11:38:44,254][root][INFO] - Using training device cuda:0.
[2022-11-01 11:38:44,254][root][INFO] - Using env device cuda:0.
[2022-11-01 11:38:44,264][root][INFO] - CWD:/tmp/arm_test/take_lid_off_saucepan/ARM
[2022-11-01 11:38:44,264][root][INFO] - Starting seed 0.
[2022-11-01 11:38:44,265][root][INFO] - Creating a PrioritizedReplayBuffer replay memory with the following parameters:
[2022-11-01 11:38:44,265][root][INFO] - 	 timesteps: 1
[2022-11-01 11:38:44,265][root][INFO] - 	 replay_capacity: 100000
[2022-11-01 11:38:44,265][root][INFO] - 	 batch_size: 128
[2022-11-01 11:38:44,265][root][INFO] - 	 update_horizon: 1
[2022-11-01 11:38:44,265][root][INFO] - 	 gamma: 0.990000
[2022-11-01 11:38:44,265][root][INFO] - 	 saving to RAM
[2022-11-01 11:38:44,269][root][INFO] - Filling replay with demos...
[2022-11-01 11:38:45,682][root][INFO] - Replay filled with demos.
[2022-11-01 11:38:46,631][root][INFO] - # Q-attention Params: 86386
/home/softgear/stepjam_ARM/ARM/arm/arm/next_best_pose_agent.py:148: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)
  action_min_max = torch.tensor(self._action_min_max).to(device)
[2022-11-01 11:38:46,656][root][INFO] - # NBP Critic Params: 1085572
[2022-11-01 11:38:46,656][root][INFO] - # NBP Actor Params: 51152
/home/softgear/stepjam_ARM/ARM/launch.py:332: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name='config', config_path='conf')
/home/softgear/stepjam_ARM/ARM/launch.py:332: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name='config', config_path='conf')
/home/softgear/stepjam_ARM/ARM/launch.py:332: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name='config', config_path='conf')
/home/softgear/stepjam_ARM/ARM/arm/arm/qattention_agent.py:35: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  indices = torch.cat((m // t_shape[-1], m % t_shape[-1]), dim=1)
/home/softgear/stepjam_ARM/ARM/arm/arm/next_best_pose_agent.py:148: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)
  action_min_max = torch.tensor(self._action_min_max).to(device)
/home/softgear/stepjam_ARM/ARM/arm/arm/next_best_pose_agent.py:148: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)
  action_min_max = torch.tensor(self._action_min_max).to(device)
/home/softgear/stepjam_ARM/ARM/arm/arm/qattention_agent.py:35: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  indices = torch.cat((m // t_shape[-1], m % t_shape[-1]), dim=1)
/home/softgear/stepjam_ARM/ARM/arm/arm/qattention_agent.py:35: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  indices = torch.cat((m // t_shape[-1], m % t_shape[-1]), dim=1)
[CoppeliaSim:loadinfo]   done.
Process train_env0:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/softgear/.local/lib/python3.8/site-packages/yarr/runners/_env_runner.py", line 169, in _run_env
    raise e
  File "/home/softgear/.local/lib/python3.8/site-packages/yarr/runners/_env_runner.py", line 143, in _run_env
    for replay_transition in generator:
  File "/home/softgear/.local/lib/python3.8/site-packages/yarr/utils/rollout_generator.py", line 30, in generator
    agent_obs_elems = {k: np.array(v) for k, v in
  File "/home/softgear/.local/lib/python3.8/site-packages/yarr/utils/rollout_generator.py", line 30, in <dictcomp>
    agent_obs_elems = {k: np.array(v) for k, v in
  File "/home/softgear/.local/lib/python3.8/site-packages/torch/_tensor.py", line 757, in __array__
    return self.numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
[2022-11-01 11:38:55,800][root][WARNING] - Env train_env0 failed (1 times <= 10). restarting
[CoppeliaSim:loadinfo]   done.
^C (SIGINT)

[yananliusdu]

same errors here when trying to run ARM with a different task........

[Colin-KelinLi]

@yananliusdu @softgearko Hi guys, I also encountered this problem. I found it is because some of the tensors are on CUDA while some are on CPU when you are running ARM method code. What you need to do is convert all the data in the rollout generator from CUDA to CPU.

I solved this problem as follows:

1. Adding the following lines

act_res.action = act_res.action.cpu()
act_res.info['q_values'] = act_res.info['q_values'].cpu()    `
act_res.observation_elements['front_pixel_coord'] = act_res.observation_elements['front_pixel_coord'].cpu()

before line 415 of /path/to/your/arm/folder/arm/next_best_pose_agent.py, which is return act_res.

2. Also in this file, find the function act_summaries(self), and modify the original code of this function as:

        summaries = [
            ImageSummary('%s/crops/act/rgb' % NAME,
                         (self._act_crop_summaries[0].cpu() + 1.0) / 2.0),
            ImageSummary('%s/crops/act/point_cloud' % NAME,
                         self._act_crop_summaries[1].cpu()),
        ]

3. In /path/to/your/arm/folder/arm/qattention_agent.py, also modify the original return as:

       return [
            ImageSummary('%s/Q_act' % NAME, QAttentionAgent.generate_heatmap(
                self._act_qvalues.cpu(), ((self._rgb_observation + 1) / 2.0).cpu()).cpu())
        ]

Now it will work, enjoy it :)