NaN Control Inputs Cause Simulation Instability in Humanoid Task

[xiaochuanma]

Intro

Hi!

I am a graduate student at TUM, I use MuJoCo for my research on the unitree g1 humanoid.

My setup

I'm doing some locomotion control tasks with the humanoid, and I'm working with the mjx for rollouts of numerous smaple controls using the force.
mujoco 3.2.4
mujoco-mjx 3.2.4
python 3.12
os: Linux 6.8.0-52-generic

My question

When running the Humanoid sampling and control task, the simulation becomes numerically unstable after 0.48 seconds. Specifically, NaN values appear in actuator 0, causing the simulation to crash.

Error message:
WARNING: Nan, Inf or huge value in CTRL at ACTUATOR 0. The simulation is unstable. Time = 0.4800.

The issue appears in the control input calculation:

Step the simulation

        for i in range(sim_steps_per_replan):
            t = i * mj_model.opt.timestep
            u = controller.get_action(policy_params, t)
            qtorque = controller.task.kp * (u - mj_data.qpos[7:]) + controller.task.kv * (mj_data.qvel[6:])
            mj_data.ctrl[:] = qtorque
            mujoco.mj_step(mj_model, mj_data)
            viewer.sync()

And here is how i evaluate the rollouts:
def _scan_fn(
x: mjx.Data, u: jax.Array,
) -> Tuple[mjx.Data, Tuple[mjx.Data, jax.Array, jax.Array]]:
"""Compute the cost and observation, then advance the state."""
x = mjx.forward(model, x) # compute site positions

        # Compute the torque from joint space control actions
        qtorque = self.task.kp * (u - x.qpos[7:])  + self.task.kv * (x.qvel[6:]) 

        cost = self.task.dt * self.task.running_cost(x, qtorque)
        sites = self.task.get_trace_sites(x)

        # Advance the state for several steps, zero-order hold on control
        x = jax.lax.fori_loop(
            0,
            self.task.sim_steps_per_control_step,
            lambda _, x: mjx.step(model, x),
            x.replace(ctrl=qtorque),
        )

        return x, (x, cost, sites)

Minimal model and/or code that explain my question

If you encountered the issue in a complex model, please simplify it as much as possible (while still reproducing the issue).

Model:
mjx_issue.zip

Confirmations

• I searched the latest documentation thoroughly before posting.
• I searched previous Issues and Discussions, I am certain this has not been raised before.

[yuvaltassa]

MuJoCo is complaining that you are giving it nans. The issue is presumably in your code?