[Question] Lift Task with UR10e and Robotiq 2F-140

[Sylelil]

Hi,

I'm working on a custom lift task in Isaac Lab using the UR10e robotic arm equipped with the Robotiq 2F-140 gripper. Below, I summarize the steps I've followed so far and describe the issue I’m currently encountering.

---

Steps Completed

1. USD Asset Setup

• Created a custom USD using Isaac Sim.
• Added the UR10e arm and Robotiq 2F-140 gripper using existing Isaac Sim assets.
• Enabled the required variants: physics, sensors, and gripper.

Preview of the setup:

2. Custom Configuration

Robot Configuration

We created a custom ArticulationCfg for the UR10e with the Robotiq 2F-140:

UR10_ROBOTIQ_CFG = ArticulationCfg(
    spawn=sim_utils.UsdFileCfg(
        usd_path=f"/workspace/isaaclab/source/isaaclab_assets/isaaclab_assets/robots/ur10e_2f140.usd",
        rigid_props=sim_utils.RigidBodyPropertiesCfg(
            disable_gravity=False,
            max_depenetration_velocity=5.0,
        ),
        activate_contact_sensors=False,
    ),
    init_state=ArticulationCfg.InitialStateCfg(
        joint_pos={
            # Arm joints
            "shoulder_pan_joint": 0.0,
            "shoulder_lift_joint": -1.0,
            "elbow_joint": 1.0,
            "wrist_1_joint": 0.0,
            "wrist_2_joint": 0.0,
            "wrist_3_joint": 0.0,
            # Gripper joints (optional initial positions)
            "left_inner_finger_joint": 0.0,
            "right_inner_finger_joint": 0.0,
            "left_outer_finger_joint": 0.0,
            "right_outer_finger_joint": 0.0,
            "finger_joint": 0.0,
            "right_outer_knuckle_joint": 0.0,
            "right_inner_finger_pad_joint": 0.0,
            "left_inner_finger_pad_joint": 0.0
        },

    ),
    actuators={
        "arm": ImplicitActuatorCfg(
            joint_names_expr=[
                "shoulder_pan_joint",
                "shoulder_lift_joint",
                "elbow_joint",
                "wrist_1_joint",
                "wrist_2_joint",
                "wrist_3_joint",
            ],
            velocity_limit_sim=20.0,
            effort_limit_sim=87.0,
            stiffness=800.0,
            damping=40.0,
        ),
        "gripper": ImplicitActuatorCfg(
            joint_names_expr=[
                "left_inner_finger_joint",
                "right_inner_finger_joint",
                "left_outer_finger_joint",
                "right_outer_finger_joint",
                "finger_joint",
                "right_outer_knuckle_joint",
                "right_inner_finger_pad_joint",
                "left_inner_finger_pad_joint"
            ],
            velocity_limit_sim=20.0,
            effort_limit_sim=20.0,
            stiffness=500.0,
            damping=0.0,
        ),
    },
)

"""Configuration of UR10e arm with Robotiq 2F-140 gripper using implicit actuator models."""

Lift Environment Configuration

from isaaclab.assets import RigidObjectCfg
from isaaclab.sensors import FrameTransformerCfg
from isaaclab.sensors.frame_transformer.frame_transformer_cfg import OffsetCfg
from isaaclab.sim.schemas.schemas_cfg import RigidBodyPropertiesCfg
from isaaclab.sim.spawners.from_files.from_files_cfg import UsdFileCfg
from isaaclab.utils import configclass
from isaaclab.utils.assets import ISAAC_NUCLEUS_DIR

from isaaclab_tasks.manager_based.manipulation.lift import mdp
from isaaclab_tasks.manager_based.manipulation.lift.lift_env_cfg import LiftEnvCfg

from isaaclab.markers.config import FRAME_MARKER_CFG  # isort: skip
from isaaclab_assets import UR10_ROBOTIQ_CFG  # devi verificare il path corretto

@configclass
class UR10eCubeLiftEnvCfg(LiftEnvCfg):
    def __post_init__(self):
        super().__post_init__()

        # Set UR10e as robot with correct prim path
        self.scene.robot = UR10_ROBOTIQ_CFG.replace(prim_path="{ENV_REGEX_NS}/ur10e")

        # Set actions for UR10e
        self.actions.arm_action = mdp.JointPositionActionCfg(
            asset_name="robot",
            joint_names=[
                "shoulder_pan_joint",
                "shoulder_lift_joint",
                "elbow_joint",
                "wrist_1_joint",
                "wrist_2_joint",
                "wrist_3_joint"
            ],
            scale=0.5,
            use_default_offset=True
        )

        self.actions.gripper_action = mdp.BinaryJointPositionActionCfg(
            asset_name="robot",
            joint_names=[
                "left_inner_finger_joint",
                "right_inner_finger_joint",
                "left_outer_finger_joint",
                "right_outer_finger_joint",
                "finger_joint",
                "right_outer_knuckle_joint",
                "right_inner_finger_pad_joint",
                "left_inner_finger_pad_joint"
            ],
            open_command_expr={
                "left_inner_finger_joint": 0.04,
                "right_inner_finger_joint": 0.04,
                "left_outer_finger_joint": 0.04,
                "right_outer_finger_joint": 0.04,
                "finger_joint": 0.04,
                "right_outer_knuckle_joint": 0.04,
                "right_inner_finger_pad_joint": 0.04,
                "left_inner_finger_pad_joint": 0.04
            },
            close_command_expr={
                "left_inner_finger_joint": 0.0,
                "right_inner_finger_joint": 0.0,
                "left_outer_finger_joint": 0.0,
                "right_outer_finger_joint": 0.0,
                "finger_joint": 0.0,
                "right_outer_knuckle_joint": 0.0,
                "right_inner_finger_pad_joint": 0.0,
                "left_inner_finger_pad_joint": 0.0
            }
        )

        # End effector body name (da verificare nel tuo USD)
        self.commands.object_pose.body_name = "robotiq_base_link"

        # Cube as object to lift
        """ self.scene.object = RigidObjectCfg(
            prim_path="{ENV_REGEX_NS}/Object",
            init_state=RigidObjectCfg.InitialStateCfg(pos=[0.5, 0, 0.055], rot=[1, 0, 0, 0]),
            spawn=UsdFileCfg(
                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
                scale=(0.8, 0.8, 0.8),
                rigid_props=RigidBodyPropertiesCfg(
                    solver_position_iteration_count=16,
                    solver_velocity_iteration_count=1,
                    max_angular_velocity=1000.0,
                    max_linear_velocity=1000.0,
                    max_depenetration_velocity=5.0,
                    disable_gravity=False,
                ),
            ),
        ) """

        # Frame marker configuration
        marker_cfg = FRAME_MARKER_CFG.copy()
        marker_cfg.markers["frame"].scale = (0.1, 0.1, 0.1)
        marker_cfg.prim_path = "/Visuals/FrameTransformer"
        self.scene.ee_frame = FrameTransformerCfg(
            prim_path="{ENV_REGEX_NS}/ur10e/base_link",
            debug_vis=True,
            visualizer_cfg=marker_cfg,
            target_frames=[
                FrameTransformerCfg.FrameCfg(
                    prim_path="{ENV_REGEX_NS}/ur10e/gripper/robotiq_base_link",
                    name="end_effector",
                    offset=OffsetCfg(
                        pos=[0.0, 0.0, 0.1034],
                    ),
                )
            ],
        )

@configclass
class UR10eCubeLiftEnvCfg_PLAY(FrankaCubeLiftEnvCfg):
    def __post_init__(self):
        super().__post_init__()
        self.scene.num_envs = 50
        self.scene.env_spacing = 2.5
        self.observations.policy.enable_corruption = False

❗ Current Issue

We adapted the default Franka Lift task from Isaac Lab to work with the UR10e + Robotiq 2F-140 setup by modifying the robot and environment configurations as shown above.

We trained the policy using the rsl_rl library with 4096 parallel environments. While the simulation runs smoothly, the robot consistently fails to complete the lift task. In particular:

• The reward consistently trends toward large negative values during training.

• The training loss rapidly increases and appears to diverge toward infinity, with no sign of convergence.

Screenshot:

If you’ve worked with similar setups or have tips for:

• Stabilizing the gripper control,
• Improving reward shaping or reward thresholds,
• Validating mimic joint or articulation configs,

...any suggestions would be much appreciated!

Thanks in advance for your support!

[RandomOakForest]

Thank you for posting this post. I'll move it to our Discussions section. Here is a summary of things to consider.

1. Physics and Simulation Configuration

Robot and Gripper Articulation

• Solver Iterations: Increase the Solver Position Iterations Count (e.g., to 64) and Solver Velocity Iterations Count (e.g., to 4) for the UR10e and gripper. This improves simulation stability for complex manipulators and mimic joints¹.
• Effort Limits: Ensure joint effort (torque/force) limits for both the UR10e and the 2F-140 gripper are set appropriately. For the gripper, set the Max Force to a realistic value (e.g., 200) to avoid unrealistic or unstable behavior¹.
• Friction: Assign physics materials to the gripper fingertips with realistic static and dynamic friction coefficients (e.g., 1.0 for both) to ensure stable grasping and prevent slippage or oscillations¹.

Gain Tuning

• Use the Gain Tuner to set the natural frequency (Nat. Freq.) and damping ratio (Damping Ratio) for all joints. For critical damping, use a Nat. Freq. of 0.5 and a Damping Ratio of 1.0. Improper gains can cause instability, leading to NaNs or diverging losses¹.

2. Reward Function and Task Design

Reward Engineering

• Check for Degenerate or Unreachable Rewards: If the reward function penalizes the agent too harshly or is not achievable with the current robot/gripper setup, the agent will accumulate large negative rewards and fail to learn. Review each reward term and ensure the task is physically feasible for the UR10e + 2F-140².
• Reward Scaling: Excessively large negative rewards can destabilize policy gradients, causing loss divergence. Consider rescaling or clipping negative rewards to prevent unbounded loss values².
• Reset Logic: Ensure that environment resets do not place the robot or objects in invalid or unstable states, which can propagate NaNs and destabilize training³.

Example: Reward Function Structure

• Use modular reward terms with clear, achievable objectives (e.g., alive bonus, object grasped, object lifted, penalties for dropping or excessive force)⁴.
• Avoid reward terms that can become unboundedly negative or are always negative regardless of agent behavior.

3. rsl_rl and Training Hyperparameters

Parallel Environments

• While 4096 environments can accelerate training, it also increases the risk of instability if the simulation or reward function is not robust. Try reducing the number of environments to see if the issue persists³.

Hyperparameter Tuning

• Learning Rate: A high learning rate can cause loss divergence, especially with unstable rewards. Try lowering the learning rate.
• Gradient Clipping: Ensure max_grad_norm is set to a reasonable value (e.g., 1.0) to prevent exploding gradients⁵⁶.
• Entropy Coefficient: If the entropy coefficient is too high, the policy may become overly random, making learning unstable.

Debugging NaNs and Divergence

• Monitor for NaNs in observations, actions, or rewards. These often indicate simulation instability or invalid physics states³.
• Reduce the physics timestep or increase solver iterations if instability is observed.

4. Additional Best Practices

• Headless Training: For large numbers of environments, run in headless mode to reduce memory and computational overhead³.
• Collision Meshes: Simplify collision meshes to the minimum required for accurate contact simulation, reducing the risk of simulation artifacts³.
• Observation/Action Sanity Checks: Ensure all observations and actions are within expected ranges before feeding them to the policy network.

Summary Table

Area	Check/Action	Reference
Physics	Increase solver iterations, set realistic effort limits, tune gains	1
Gripper Friction	Assign physics materials, set friction coefficients	1
Reward Function	Avoid unachievable/degenerate rewards, rescale negatives, modular design	24
Training Params	Lower learning rate, enable gradient clipping, check entropy coefficient	56
Simulation Stability	Reduce timestep, check for NaNs, validate resets	3
Parallelism	Reduce number of environments if instability persists	3

Footnotes

1. https://docs.isaacsim.omniverse.nvidia.com/latest/robot_setup_tutorials/tutorial_configure_manipulator.html ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

2. https://isaac-sim.github.io/IsaacLab/main/source/setup/walkthrough/training_jetbot_reward_exploration.html ↩ ↩² ↩³

3. https://isaac-sim.github.io/IsaacLab/main/source/overview/reinforcement-learning/training_guide.html ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

4. https://isaac-sim.github.io/IsaacLab/main/source/overview/core-concepts/task_workflows.html ↩ ↩²

5. https://isaac-sim.github.io/IsaacLab/main/source/api/lab_rl/isaaclab_rl.html#module-isaaclab_rl.rsl_rl ↩ ↩²

6. https://isaac-sim.github.io/IsaacLab/main/_modules/isaaclab_rl/rsl_rl/rl_cfg.html ↩ ↩²

[Sylelil]

Hi @RandomOakForest , thank you very much for the helpful and detailed response. I'm currently studying the topic in more depth, and your pointers have provided a solid direction to proceed.

Here’s a summary of what I’ve updated and tested so far based on your suggestions:

1. Physics and Simulation Configuration

Robot and Gripper Articulation

• Solver Iterations: Increased position and velocity solver iterations to 64 and 4, respectively.

• Friction Coefficients: Assigned a PhysicsMaterial with both static and dynamic friction coefficients set to 1.0 on the gripper fingertips.

• Effort Limits: Set the 2F-140 gripper’s max force to 200.

Gain Tuning

I explored the Gain Tuner panel but couldn’t find any options to directly configure the Natural Frequency or Damping Ratio parameters for the joints.
Below is a screenshot of the Gain Tuner interface in Isaac Sim version 4.5.0:

According to the official documentation (Configure Joint Effort Limits), all joints except the finger joint should be mimic joints. However, by inspecting the stage in my USD, it appears that only the right outer knuckle joint is actually defined as a mimic joint. For this joint, I was able to access and modify the Natural Frequency and Damping Ratio parameters directly in the Physics tab.

The view from the Physics Inspector seems to confirm this and highlights a discrepancy compared to the setup described in the documentation:

Robot Configuration

I updated the ArticulationCfg to include only the finger_joint as the actuator for the gripper, following the setup demonstrated in the NVIDIA Omniverse Hands-on Lab Tutorial. I also enabled the enabled_self_collisions flag to allow for more realistic physical interactions between the robot’s links.
Additionally, I manually tuned the damping and stiffness parameters for both the UR10e arm and the 2F-140 gripper to enhance simulation stability.

Here is the code with the updates:

import math
import isaaclab.sim as sim_utils
from isaaclab.actuators import ImplicitActuatorCfg
from isaaclab.assets.articulation import ArticulationCfg

##
# Configuration
##

  
UR10_ROBOTIQ_CFG = ArticulationCfg(
    spawn=sim_utils.UsdFileCfg(
        usd_path=f"/workspace/isaaclab/source/isaaclab_assets/isaaclab_assets/robots/ur10e_2f140.usd",
        rigid_props=sim_utils.RigidBodyPropertiesCfg(
            disable_gravity=False,
            max_depenetration_velocity=5.0,
        ),
        activate_contact_sensors=True,
        articulation_props=sim_utils.ArticulationRootPropertiesCfg(
            enabled_self_collisions=True, solver_position_iteration_count=8, solver_velocity_iteration_count=0
        ),
    ),
    init_state=ArticulationCfg.InitialStateCfg(
        joint_pos={
            # Arm joints
            "shoulder_pan_joint": 0.0,
            "shoulder_lift_joint": -1.712,
            "elbow_joint": 1.712,
            "wrist_1_joint": 0.0,
            "wrist_2_joint": math.pi/2,
            "wrist_3_joint": 0.0,
            # Gripper joints (optional initial positions)
            "finger_joint": 0.04,            
        },
    ),
    actuators={
        "arm": ImplicitActuatorCfg(
            joint_names_expr=[
                "shoulder_pan_joint",
                "shoulder_lift_joint",
                "elbow_joint",
                "wrist_1_joint",
                "wrist_2_joint",
                "wrist_3_joint",
            ],
            velocity_limit_sim=0.5,
            effort_limit_sim=87.0,
            stiffness=300.0,
            damping=20.0,
        ),
        "gripper": ImplicitActuatorCfg(
            joint_names_expr=[
                "finger_joint",                
            ],
            velocity_limit_sim=10.0,
            effort_limit_sim=200.0,
            stiffness=300.0,
            damping=10.0,
        ),
    },
)
"""Configuration of UR10e arm with Robotiq 2F-140 gripper using implicit actuator models."""

2. Reward Function and Task Design

For the reward function and reset logic, I am currently using the Franka robot setup as a baseline, as detailed here.

3. rsl_rl and Training Hyperparameters

Using the updated setup, I launched a new training experiment with the rsl_rl library, running 400 parallel environments. The following hyperparameters were manually tuned:

• max_iterations = 2000
• num_steps_per_env = 32
• actor_hidden_dims = [512, 256, 128]
• critic_hidden_dims = [512, 256, 128]
• entropy_coeff = 0.01
• learning_rate = 3e-4
• num_mini_batches = 8
• gamma = 0.99

As shown in the video below, the robot is now able to grasp the cube. However, it still struggles to achieve the final objective — performing a consistent lift under varying conditions. I also ran additional training sessions using the same setup but wasn’t able to consistently reproduce the successful behavior, which suggests that the policy may still lack robustness.

lift_task_play.mp4

Below are the plots showing the reward progression and loss trends during training:

Based on all the modifications described above, I’m now looking for some guidance on the following key aspects of the implementation:

1. USD File and Mimic Joints:

   • In my USD file, only one joint — the right outer knuckle joint — is currently defined as a mimic joint. Could this indicate an incomplete configuration of the Robotiq 2F-140 gripper?

   • In the ArticulationCfg, I’ve only defined "finger_joint" as the actuator for the gripper. Is that sufficient to control the gripper effectively, or should other joints be explicitly included in the actuator list as well?

2. Gain Tuner and Joint Parameters:
   I was only able to modify the Natural Frequency and Damping Ratio for the mimic joint. Could this suggest a misconfiguration in the USD file for the other joints?

3. Reward Function and Reset Logic:
   I’m currently using the reward and reset structure from the Franka Lift environment as a baseline. Is this a reasonable baseline for my UR10e + 2F-140 setup, or would you recommend adapting it further to better suit this task?

4. Training Stability and Policy Robustness:
   As seen in the training plots and video, the robot can now grasp the cube, but it still struggles to consistently complete a stable lift. Do you have any practical suggestions to improve training stability, enhance policy generalization, and ensure the robot can reliably complete the task?

Any feedback — especially on how to fix, optimize, or rethink the setup described above — would be extremely valuable. I’ll continue experimenting and will share new results as they come in.

Thanks again for the time and support!

[Viru97]

Hii Sylelil!

I am also facing similar kind of issue, did you change the gripper usd file to make rest of the joints mimic joints?
Thanks.

[FHunist]

Hi Sylelil,

I also face the same issue of the goal tracking being really poor.
If you have any update, let me know! If I figure something out I'll update as well.
Thanks!