Insight for training a small bipedal robot using Mujoco Playground

[ParadoxRobotics]

I'm Quentin Munch, a robotics and computer vision engineer. During my free time, I've been developing a small 3D printed bipedal robot that looks like the BDX droid from disney research.
My BD-5 uses the same dynamixel servos as the Robotis OP3 and it's equipped with a 9Dof IMU and Raspberry Pi zeros 2W.

I managed to integrate the robot in the mujoco playground framework and train it to obtain a walking policy following the berkeley / T1 / OP3 joystick task and description.
However, my current policy (see video attachment) is not very optimal : the robot doesn't lift its feet and the backward walk is faster than the forward one.
In this matter, I have the following questions :

• Is it recommended to use the feet clearance cost + feet phase ? or the feet air time + feet slip + feet phase ? (To avoid rapid shuffle and help lifting the feet). I noticed a lot of drag of the back of the feet, should I create a reward that enforce the feet orientation to remain parallel to the ground ?
• In my model, the collision of the ground/foot is the downsampled stl file of the foot, should I use a collision cylinder instead ?
• Do you think I need to integrate a reference motion during training ? Since it's tedious to specify the weights of the reward, I was thinking of building a (command dependent) basic gait generator combined with mujoco mink to compute the IK. The resulting feet pose, body pose and joint position is then used in an imitation cost like the original BD-X. But I've also read that simple periodic reward composition can induce a stable walking policy...So I don't know if it's useful.
• What method or approach do you recommend for reference motion based training ?
• I've also added in the foot phase cost computation, a linear velocity mask similar to the Unitree G1. What motivates this modification in the G1 code compared to the other joystick task ?
• Finally, is there a method for reward weight tuning ? Like tuning the action scale first then the torque...etc

My code is a fork of the original repo, the bd5 is in _src/locomotion folder if you want to look. Also I've modified play_joystick code in the experimental/sim2sim folder.
link : https://github.com/ParadoxRobotics/mujoco_playground

Thank you for your help !

Best regards,

Quentin.

prettygoodpolicy.webm