How to define a keyframe for joints in bodies that are being replicated? #2360

adlarkin · 2025-01-15T04:44:17Z

adlarkin
Jan 15, 2025

Intro

Hello, I'm a graduate student and use MuJoCo for my research on robotics.

My setup

MuJoCo version 3.2.6 (python - pip install mujoco)

MacOS 15.2, arm architecture (apple M3 chip)

My question

I am trying to spawn two of the same robot arm into a scene for bimanual manipulation. I plan to do this via the replicate tag. I would like to specify a keyframe so that I can initialize both robot arms to a pre-defined "home" joint configuration. However, it seems like replicate also changes the keyframes associated with the model, so I've had a hard time achieving the behavior I want.

The xml below gives a minimal example of a body with 1 DOF. I spawn two bodies with replicate, so the model has 2 DOFs. I have tried two approaches in the xml:

Make a keyframe with a qpos size of 1: <key name="home" qpos="0.5"/>. Using this, I would expect for a replicated keyframe with qpos = [0.5, 0.5]
Make a keyframe with a qpos size of 2: <key name="home" qpos="0.5 0.25"/>. Since this keyframe is already of size 2, and there are 2 joints after replicating, I would expect to be able to use this keyframe as defined in the xml.

I would like to take approach number 2 above, since I know I'll have two joints and want to have values defined for each joint. However, when I try both approach 1 and 2 listed above, I get the following behavior:

mjModel now has three keyframes: home, home0, home1
home0 and home1 seem to be associated with the replicated bodies
home is now reset to have a qpos of all 0's, which is unexpected

For this simple example, how can I achieve a keyframe with qpos = [0.5, 0.25]? Can this be done in the xml, or does it have to be set programatically?

Also, I am a little surprised that when using replicate for this example, I can define a keyframe with a qpos size of either 1 or 2 and load both of these xml variants into a model without issues. Is this expected behavior, or a bug?

Minimal model and/or code that explain my question

If you encountered the issue in a complex model, please simplify it as much as possible (while still reproducing the issue).

Model:

minimal XML

<mujoco model="2 balls translating along xy-plane">
  <worldbody>
    <replicate count="2" offset="0 1 0">
      <body name="ball" pos="0 0 1">
        <joint name="ball_slide_x" type="slide" axis="1 0 0" range="-2 2"/>
        <geom type="sphere" size="0.1" rgba="0 1 0 1"/>
      </body>
    </replicate>
  </worldbody>

  <keyframe>
      <!--
        Can also change the keyframe to the following, and then run the code snippet below (behavior doesn't change):
        <key name="home" qpos="0.5"/>
      -->
      <key name="home" qpos="0.5 0.25"/>
  </keyframe>
</mujoco>

Code:

import mujoco

model = mujoco.MjModel.from_xml_string(xml)

# I would like this to be what's defined in the xml: [0.5, 0.25]
print(model.keyframe('home').qpos)

# I assume these are added as an artifact of using `replicate`?
print(model.keyframe('home0').qpos)
print(model.keyframe('home1').qpos)

Confirmations

I searched the latest documentation thoroughly before posting.
I searched previous Issues and Discussions, I am certain this has not been raised before.

Answered by adlarkin

Jan 15, 2025

Thanks, I am able to achieve what I want programmatically by using the following xml file (I removed the <keyframe> section and added <size nkey="1"/> to manually reserve one keyframe) and python code:

Model:

manual keyframe allocation xml

<mujoco model="Ball translating along xy-plane">
  <!-- Reserve space for a "home" keyframe -->
  <size nkey="1"/>

  <worldbody>
    <replicate count="2" offset="0 1 0">
      <body name="ball" pos="0 0 1">
        <joint name="ball_slide_x" type="slide" axis="1 0 0" range="-2 2"/>
        <geom type="sphere" size="0.1" rgba="0 1 0 1"/>
      </body>
    </replicate>
  </worldbody>
</mujoco>

Code:

import mujoco

model = mujoco.MjModel.from_xml_string(…

View full answer

quagla · 2025-01-15T10:54:00Z

quagla
Jan 15, 2025
Collaborator

Hi, yes you have to set it programmatically. And yes, the fact that you can load a keyframe of size 2 sounds like a bug.

There are a few examples here https://github.com/google-deepmind/mujoco/blob/main/test/user/user_api_test.cc (look in particular at AttachSame) of the intended behavior of keyframes with attach (replicate is a self-attach).

0 replies

adlarkin · 2025-01-15T19:07:16Z

adlarkin
Jan 15, 2025
Author

Thanks, I am able to achieve what I want programmatically by using the following xml file (I removed the <keyframe> section and added <size nkey="1"/> to manually reserve one keyframe) and python code:

Model:

manual keyframe allocation xml

<mujoco model="Ball translating along xy-plane">
  <!-- Reserve space for a "home" keyframe -->
  <size nkey="1"/>

  <worldbody>
    <replicate count="2" offset="0 1 0">
      <body name="ball" pos="0 0 1">
        <joint name="ball_slide_x" type="slide" axis="1 0 0" range="-2 2"/>
        <geom type="sphere" size="0.1" rgba="0 1 0 1"/>
      </body>
    </replicate>
  </worldbody>
</mujoco>

Code:

import mujoco

model = mujoco.MjModel.from_xml_string(xml)
data = mujoco.MjData(model)

print(data.qpos) # should print [0, 0]
model.key_qpos[0] = np.array([0.5, 0.25])
mujoco.mj_resetDataKeyframe(model, data, 0)
print(data.qpos) # should print [0.5, 0.25]

This approach has the following limitations

Must manually set things like model.key_qpos before using the keyframe
I don't think I'm able to assign a string name to this keyframe. A keyframe with no name means that to retrieve keyframe info, I have to index the keyframe fields in mjModel manually instead of calling model.keyframe('keyframe_name').id.

I can work around these limitations for now with helper python code/configuration files/etc. Let me know if there's a better way to programmatically set/use keyframes than what I outlined here.

And yes, the fact that you can load a keyframe of size 2 sounds like a bug.

Here's a simpler model file that shows a potential bug with replicate/keyframe sizes:

Model:

buggy replicate keyframe xml

<mujoco model="Test replicate and keyframe sizes">
  <worldbody>
    <replicate count="1" offset="0 1 0">
      <body name="ball" pos="0 0 1">
        <geom type="sphere" size="0.1"/>
      </body>
    </replicate>
  </worldbody>

  <keyframe>
    <!-- this should not work since the model has no joints! -->
    <key name="home" qpos="0.5 0.25"/>
  </keyframe>
</mujoco>

The model loads without errors if replicate is left in the xml. If replicate is removed from the xml, I get this error when I try to load the model, as expected:

ValueError: Error: keyframe 0: invalid qpos size, expected length 0

Would you like me to submit a bug report as a separate issue for this?

2 replies

yuvaltassa Jan 15, 2025
Maintainer

Yes, please open a bug for this, thanks 🙂

adlarkin Jan 17, 2025
Author

I opened an issue: #2365

I'll mark my response as an answer to my original question. Thank you!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to define a keyframe for joints in bodies that are being replicated? #2360

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

How to define a keyframe for joints in bodies that are being replicated? #2360

Uh oh!

Uh oh!

adlarkin Jan 15, 2025

Intro

My setup

My question

Minimal model and/or code that explain my question

Confirmations

Replies: 2 comments · 2 replies

Uh oh!

quagla Jan 15, 2025 Collaborator

Uh oh!

adlarkin Jan 15, 2025 Author

Uh oh!

yuvaltassa Jan 15, 2025 Maintainer

Uh oh!

adlarkin Jan 17, 2025 Author

adlarkin
Jan 15, 2025

Replies: 2 comments 2 replies

quagla
Jan 15, 2025
Collaborator

adlarkin
Jan 15, 2025
Author

yuvaltassa Jan 15, 2025
Maintainer

adlarkin Jan 17, 2025
Author