Feature Request: Support for robots using the Robotiq 2F85 Gripper.

[Tass0sm]

Is your feature request related to a problem? Please describe.
I am trying to implement an OmniGibson class for a UR5e manipulator robot with a Robotiq 2F85 Gripper. After having set up the gripper USD, loading it in OmniGibson is forbidden by the following assertion in entity_prim.py

assert all(
            [in_degrees[node] == 1 for node in G.nodes if node != self.root_link_name]
        ), "All non-root links should have in-degree of 1!"

Describe the solution you'd like
I might be misunderstanding, but I think this assertion implies that closed kinematic chains with non-articulated joints are unsupported, and adding support for these using non-articulated joints would be the ideal solution.

Describe alternatives you've considered
I am still working on understanding what is required within the OmniGibson code to support this. While testing, I removed this assertion and also was unable to proceed due to the two following assertions in joint_prim.py and entity_prim.py.

assert self._prim.HasAPI(
                lazy.pxr.PhysxSchema.JointStateAPI
            ), "Revolute or Prismatic joints must already have JointStateAPI added!"

Edit: after adding the JointStateAPI to the new joints in the USD file, this assertion is no longer an issue.

assert self.n_joints == len(self._joints), (
            f"Number of joints inferred from prim tree ({self.n_joints}) does not match number of joints "
            f"found in the articulation view ({len(self._joints)})!"
        )

Another alternative may be creating a simpler model that approximates the behavior of this particular gripper without the complicated articulation.

Additional context
I'm attaching the USD file I'm using and the file for the UR5e class that is relatively portable.

ur5e_for_issue.txt

USD File: https://drive.google.com/file/d/1iVdskWSU99QxwzQvCT3XiJe46zlLYLLg/view?usp=drive_link

[Tass0sm]

I believe I have resolved this issue. I added necessary checks for the "physics:excludedFromArticulation" property in the construction of the articulation tree and the calculation for the number of joints. I also have removed the in_degrees assertion temporarily and would replace it with a different assertion to ensure that there are no closed loops. I will close this issue once I get the entire robot working.

Edit: it looks like the next step will require implementing support for mimic joints

[Tass0sm]

I've decided to remove the mimic joints from the model since they were causing stability issues for the physics solver. Now the model is finished and uses several non-driven joints for the gripper and two driven joints. With the 6 DOF from the arm, the model should have 8-DOF in total. However the 12 joints make Omnigibson treat it as 12-DOF. I'm now working to fix this.

Edit: Although treating the gripper as 6-DOF when it is only 2-DOF is conceptually incorrect, it seems like I can still proceed with using the robot as long as I provide 12-DOF configurations during reset. This is good enough for my purpose so I'm moving past this issue and now working on control.

[Tass0sm]

My UR5e with Robotiq 2F 85 Gripper is now controllable. I will leave this open in the case that the other problems mentioned above need further discussion or if the maintainers are interested in this robot or others with the Robotiq 2F85 being added to the project.

[hang-yin]

Thank you so much for investigating this!!! Tagging @cremebrule fyi

[martin-sedlacek]

Hi all, I am also trying to simulate this gripper but with a Panda arm. After a lot of trial and error I managed to replace the gripper on the FrankaMounted USD and can now control both the arm and the gripper. Similarly to what @Tass0sm said, I also had to modify the reset joint state and comment out some checks.

I will just preface that this is my first time using OmniGibson/isaac and I do not have much experience with simulation, so it is currently very clunky, but is somewhat working.

I mostly relied on this blogpost from NVIDIA for setting up the joint drives etc. in the USD: https://docs.omniverse.nvidia.com/isaacsim/latest/advanced_tutorials/tutorial_advanced_rigging_complex_structures.html

Actual USD file I used:
https://drive.google.com/file/d/1rFC-xwK_nXBnYWNlQpel0RY0iUczkNB8/view?usp=sharing

However, the gripper is unable to actually move / exert any force on objects, resulting in behaviour like this:

debug_gripper_grasp.mp4

debug_gripper_side_push.mp4

I tried tuning the force parameters but this seems to have no effect on the issue.

I also tried the same scene config with the default FrankaMounted, which worked, so the object is definitely movable.

I an now basically stuck on this as I am unable to pinpoint what is causing the behaviour. If you have any suggestions for debugging this, I would really appreciate it. cc @hang-yin @cremebrule

[martin-sedlacek]

Update: managed to somewhat solve the issue recently - the physics fro this gripper seem to be not well behaved. I had to increase the default frequency x80 to even get any gripper-object interaction (issue in videos above) when the fingers are driven by the upper two revolute joints of the Robotiq gripper.

I remodeled it to use linear drives for the inner fingers and excluded all other joint from articulation + changing some collision meshes. Now the outer parts of each finger are basically dragged by this bottom inner finger link and the system seems to behave the same as the linear drive gripper on FrankaMounted.