Trying to locate dockerdata files and full train dataset prep

[HoustonHuff]

Hello, first of all thank you for your work on this repo.

I've been trying to get the training pipeline working and I'm finding there are multiple files missing that point to within a dockerdata directory but I'm not able to find any instructions on how to retrieve them within the repo. I was able to find most of the original models from huggingface itself but I don't know what can be done for parameter files such as /dockerdata/Moore/reference_unet.pth. I'd appreciate any information on how I can get access to the necessary files so I can begin the training.

I also saw that the data preparation seems to be done for a single video file in the context of inference rather than for a full raw video set for training. Is there a dataset file structure you'd recommend for adapting that data prep to the full videos of say HDTF?

[tiankuan93]

1. You can download Moore's pre-trained model from https://huggingface.co/patrolli/AnimateAnyone/tree/main.
2. For training data organization, please refer to https://github.com/tencent-ailab/V-Express/tree/main/scripts/prepare_dataset#training-data for data preparation.

[HoustonHuff]

I've seen these resources, I've been able to get the unet model files and I've made a looping bash script to run extract_audio_embeddings.py and extract_face_info.py through my raw video data and organize the output according to the file structure in the documentation.

However, that still leaves me with files missing, in particular the meta files that try to be loaded in by talking_face_video.py for the data loading. The list of them are here from the stage1.yaml config:

  meta_paths:
    - 'hdtf_3kps.json'
    - 'part_01_wo_low_res_wo_low_syncnet_conf.json'
    - 'p2.json'
    - 'p3.json'

I've looked my hardest for them but I haven't had any luck so far.

[tiankuan93]

Here, "*.json" refers to the prepared training data (stored in JSON format), which can be one or more files. This stage1.yaml file provides a simple example.