Translation approach -- Touchdesigner implementation?

[crismollee]

Hi! Thank you for your work on this! I am trying to create a similar tool in Touchdesigner and I was wondering how you approached the conversion of the models;

• xyz into rotational data (like the standard mixamo animation)
• different skeleton types
  Could you shine a light on your technique? Did you first convert the mediapipe skeleton into the mixamo one, or alter the mixamo skeleton? Did you run into any scale or hierarchy issues? And did you have to convert your positional data into rotational data or did you get it to work with the xyz info?
  I would also be super interested in seeing how to implement this tool into Touchdesigner, perhaps there could be a way to stream the modified animation data to a mixamo fbx Touchdesigner or another program?

Best,
Cris

[Nor-s]

I generated the spine points using simple linear interpolation from the skeleton of the mediapipe.

Here you can see the skeleton mapping between Mediapipe and Mixamo.

Anim/mp_mocap/mixamo_data.py

Line 120 in f87b413

def get_mixamo_pose_info():

[Nor-s]

First, I normalized the model information.

I resized the sizes of the bind poses to -1 ~ 1, like mediapie, while maintaining the ratio.

Anim/mp_mocap/model_node.py

Line 178 in f87b413

def normalize(self, mixamo_list, visibility_list, len = 0.0):

like here image

[Nor-s]

I explored the normalized character model root with the dfs algorithm.

I was able to obtain the rotation values ​​by applying the IK algorithm at each step.

[Nor-s]

I think it's hard to use the current implementation elsewhere.

I'm sorry, it's too dependent on my own tools.

However, you can easily find open source blender version. I don't know much about Touchdesigner tool, so I can't answer you.

[crismollee]

That's impressive, thank you for explaining! So calculating missing points, normalising the scale of the model and finally calculating the rotations with IK. Sound like a nice puzzle to attempt to rebuild in Touchdesigner :) But I'll also take a look at the blender tools to see if there might be a simpler implementation possible. Really appreciate your input, thanks again!