Issues with Franka Sim2Real with ROS

[pmfaustino]

Hello,

We are working on a Sim2Real example with the Franka Emika Panda robot, where it picks up an object and lifts it to a goal position using ROS. Our approach follows the Kuka LBR iiwa real-world example from the SKRL documentation:
🔗 Real-world Examples - SKRL. However, results have not been great.

What We Did:

1. Training in Simulation:

• Used Omniverse Isaac Lab to train the robot.
• Achieved very good results: the robot successfully lifted the object and brought it close to the goal position.

2. Adapting the ROS Environment:

• Used the Kuka LBR iiwa ROS example as a template.
• Modified the ROS environment script:
  - Adjusted the robot configuration to match the Panda.
  - Updated observations, rewards, resets, and done conditions to align with the trained policy.
• Adjusted the evaluation script to ensure it matches the same parameters as the Isaac Lab training setup.
• Created a basic Isaac Sim environment, similar to the one in Isaac Lab.

The Issue:
When running the evaluation script with ROS, the robot's behavior differs significantly from its behavior during training.

• If we play the checkpoint directly in Isaac Lab, the robot is able to pick up the object and lift it to the goal position successfully.
• However, when using the entire setup with ROS, the robot does not act the same way being unable to pick the object most of the times.

Attached Files:
Franka_Sim2Real.zip

Isaac Lab Task: franka_object_env.py
Environment (ROS): franka_task.py
Evaluation script: config.py
Checkpoint: agent.pt
Isaac Sim Environment: franka_rl.usd

We have attached the scripts used in our setup. Any insights or suggestions would be greatly appreciated!

Thanks in advance!

[ozhanozen]

How is the real robot's behavior currently?

I have worked on sim2real for the Isaac Lab lifting task with the fr3 robot for some time. The real robot's behavior did not match the simulation one initially, resulting in failures during grasping. The biggest gap in my case was arising from the mismatch in gripper speed. While my setup differed from yours quite a lot, your issue could also be gripper related.

In simulation, the gripper is very fast, and the policy somehow learns to exploit this. For example, it doesn't need to slow down near the gripping location as it doesn't need to. In the real robot, the gripper is quite slow, but since the robot doesn't know this, it doesn't slow down during the approach, hence missing the object.

In your code:

"panda_hand": ImplicitActuatorCfg(
    joint_names_expr=["panda_finger_joint.*"],
    effort_limit=200.0,
    velocity_limit=0.1, # was 0.2
    stiffness=1000000.0,
    damping=100.0,
),

Please note that in the last version of the IsaacLab, velocity_limit is not used for ImplicitActuator. You should rather use velocity_limit_sim and make it slow enough to match reality. Actually, this hindered the learning a lot in our case; the robot even learned to lift the object without using the gripper at all! For this, we solve the issue but starting with a high velocity_limit_sim and then lowering it with curriculum.

Hope that this helps!

[pmfaustino]

Sorry for bothering you again @ozhanozen

Following up on my previous answer, we have changed the code to use velocity_limit_sim instead and have decreased the speed of the gripper from 0.1 to 0.015 and the behavior seems to be similar. We have also adjusted some other simulation parameters that were incorrect.

Do you believe the speed of the gripper is still too high or could it be another unrelated issue?

Thanks in advance

[ozhanozen]

No worries.

I think 0.015 is low enough, actually it was the lower bound of what we have trained with. However, rather than fixing the velocity limit, I suggest that you do domain randomization for that: randomize the gripper velocity in a range (e.g., [0.015-0.15]) at every episode. It is very hard to know the exact value, it is better that the network is able to adapt to different values. Importantly, provide the gripper position as an observation if you didn't do it yet.

That being said there could be other unrelated issues as well. Do you also randomize the controllers gains, and damping/friction?

It is hard to say what else might be the reason for the sim2real gap. If you could share the videos of the real and simulated robots, maybe we can get some cues.

[pmfaustino]

We have not yet tried to use a dynamic velocity limit as you mentioned, but we can try it and see if the results improve. Furthermore, we also have static values for stiffness and damping, and the values we've been using are the default values when importing the robot into the simulation, but we have assured that these values match both in the Isaac Lab training and the ROS evaluation.

Until we have some updates on the results, following the changes you suggested, these videos show what our current results look like, which might help to understand the issue.

Isaac Lab Training:
Screencast from 07-04-2025 16_22_45.webm

Ros Evaluation:
Screencast from 07-04-2025 16_20_48.webm

[ozhanozen]

I think I understood your post wrong. I initially thought you were using ROS to control a real robot. After watching your ROS video and re-reading your description, I realized you are using ROS as an intermediate layer but still controlling a simulated robot on IsaacSim.

Is this correct? If so, I don't think the gripper velocity is a problem here, as the speed limit is whatever it is in the simulation; there is no difference between the two cases.

The problem might lie in potential implementation differences/mistakes between your training and evaluation/ros scripts. I had a quick look at your scripts and found these particularities:

• Your training script integrates the joint displacements on the previous desired targets (i.e., like the PLAI method), while your ros version integrates them on the current joint positions (the conventional method):
  targets = self.robot_dof_targets + self.robot_dof_speed_scales * self.dt * self.actions * self.cfg.action_scale
  vs
  joint_positions = self.robot_state["joint_position"] + (self.robot_dof_speed_scales * self.dt * action * self.action_scale).
• On your training script, object_position is in the robot root/base frame, end_effector_pos is in the world frame, and ee_object_distance is in the world frame. On the other side, on your ros script, object_pos(ition) is in the world frame, end_effector_pos is in the robot root/base frame, and ee_object_distance might be invalid as you are combining two different frames to calculate it. Just so you know, if your robot root/base frame is not the same as the world frame, these do not necessarily match.
• On your training script, I lost track of how you calculated goal_position; why do you use object position for that? Did you want to use self._robot.data.root_state_w instead? Anyway please note thatself._object.data.root_pos_w is in the world frame, which is mostly named object_pos_w. object_pos_b you see commonly is the version that is wrt the robot root state.
• In general, I suggest that you convert everything to the robot root/base frame (e.g., wrt the robot.data.root_pos_w), to be consistent (and name them xxx_b).

Please take a look at these if they have escaped your attention. I hope these will solve the problem.

Why do you use ros to control the robot in the simulation btw? Is this a preliminary step before testing on the real robot?

[pmfaustino]

Thank you for the follow-up and I apologize for the delayed answer. Yes, you are absolutely right. Our end goal is to test it using a real robot, but for now we have just been following the "Kuka LBR iiwa Real-World (ROS/ROS2)" example to make sure everything is good before shifting our focus to the real robot. Since it's inside the Real-world examples tab I thought the most appropriate tag for this discussion would be sim2real, but I could be mistaken though.

Either way, I appreciate you taking the time to review our code. At the time of the initial post the code did have some issues (like the distances being in different frames as you mentioned) and some of them we were able to detect and correct in the meantime. So that code is not our most updated one anymore, that's entirely on me. I will upload our latest code below, with some of those issues already fixed, but I will still take a look at your other suggestions and will try and see if they improve our results. Specially the first change you suggested, regarding joint displacements, since that could make a huge difference on our results.

Once more, thanks for taking the time to look at our scripts and to help us, we really appreciate it. I will update this post once we're able to test the updated script with the suggested changes.

Updated ROS script (it's a python file, just renamed to txt in order to be able to upload it here):
franka_task.txt