Jerky Arm Motion When Teleoperating with Meta Quest 3

[PeterNg15]

We are experiencing latency issues when teleoperating with the Meta Quest 3. While finger movements remain smooth, arm movements are noticeably jerky, and we have not been able to achieve the fluid motion shown in the reference video. Here is a video of our robot being teleoperated: https://drive.google.com/file/d/1yr2ylrjtWPyJTt8gRIHha2g-_F3nfOFo/view?usp=sharing

Connection setup:

• User PC (host) connected via Ethernet to the same dedicated network as the G1 (no other devices present), which is also connected via Ethernet. Meta Quest 3 is connected to the same network via WiFi
• We ran the following command on the User PC: python teleop_hand_and_arm.py --xr-mode=hand --arm=G1_29 --ee=dex3

Pinging the G1 from the User PC shows low latency.

(base) user@hostpc:~/xr_teleoperate/teleop$ ping 192.168.123.165
PING 192.168.123.165 (192.168.123.165) 56(84) bytes of data.
64 bytes from 192.168.123.165: icmp_seq=1 ttl=64 time=0.651 ms
64 bytes from 192.168.123.165: icmp_seq=2 ttl=64 time=0.505 ms
64 bytes from 192.168.123.165: icmp_seq=3 ttl=64 time=0.578 ms
…
64 bytes from 192.168.123.165: icmp_seq=18 ttl=64 time=0.816 ms
64 bytes from 192.168.123.165: icmp_seq=19 ttl=64 time=0.591 ms

Pinging the Meta Quest 3 from the User PC shows latency between 80–300 ms.

(base) user@hostpc:~/xr_teleoperate/teleop$ ping 192.168.123.196
PING 192.168.123.196 (192.168.123.196) 56(84) bytes of data.
64 bytes from 192.168.123.196: icmp_seq=1 ttl=64 time=79.5 ms
64 bytes from 192.168.123.196: icmp_seq=2 ttl=64 time=6.40 ms
64 bytes from 192.168.123.196: icmp_seq=3 ttl=64 time=334 ms
64 bytes from 192.168.123.196: icmp_seq=5 ttl=64 time=74.1 ms
…
64 bytes from 192.168.123.196: icmp_seq=22 ttl=64 time=362 ms
64 bytes from 192.168.123.196: icmp_seq=23 ttl=64 time=78.2 ms
64 bytes from 192.168.123.196: icmp_seq=24 ttl=64 time=101 ms

We’re wondering if the latency to the Meta Quest 3 is contributing to the issue, and whether this is a known behavior or limitation of the device. We have tested on two different User PCs and observed the same problem. What possible solutions should we consider? Would using a wired connection with ADB reverse porting be necessary?

[silencht]

hi.

1. Did you observe this issue during the simulation deployment?
2. Please restart both your Quest 3 and your router, then try again.
3. Make sure your Wi-Fi is set to 5 GHz only; avoid combined dual-band or 2.4 GHz modes.
4. If steps 1 and 2 don’t help, try testing with a different router.
5. Although I doubt this is the cause, please still verify the following code to ensure the IK solver completes in < 10 ms. (You will need to set the logger_mp.basic_config level to DEBUG.)
6. I believe ADB is only meant to address certificate issues; it’s unlikely to have much impact on data-transfer speed.
   

   xr_teleoperate/teleop/teleop_hand_and_arm.py

   Lines 275 to 280 in c12cf6a

   	# solve ik using motor data and wrist pose, then use ik results to control arms.
   	time_ik_start = time.time()
   	sol_q, sol_tauff = arm_ik.solve_ik(tele_data.left_arm_pose, tele_data.right_arm_pose, current_lr_arm_q, current_lr_arm_dq)
   	time_ik_end = time.time()
   	logger_mp.debug(f"ik:\t{round(time_ik_end - time_ik_start, 6)}")
   	arm_ctrl.ctrl_dual_arm(sol_q, sol_tauff)