Visualization

[deeplylearner]

Can you describe the steps of the visualization below, I've completed the testing session, but I can't get through the visualization program.

[elliottwu]

After running the test script, it should create a folder with intermediate results. This line in the visualization script loads these results:

DOVE/scripts/render_visual.py

Line 468 in 61e128f

(prior_shape, seq_shape, frame_shape, pose, arti_params, mask_gt, input_image, tex_im) = (load_seq_files(osp.join(res_dir, seq_folder), sfx, frame_stride=frame_stride).to(device) for sfx in sfxs)

.

You should be able to start tracing the issues from here.

[deeplylearner]

Thank you for your answer, I've completed the training, testing, and visualization process. But I wondered, how do I use my own single image for testing and visualization?

[elliottwu]

You need to crop the images around the instances first, similar to the examples in the provided datasets. You may do so manually or use an automatic segmentation model.

Then, prepare the data in the same way as the provided test images. One example you can refer to is the toy bird dataset: https://github.com/elliottwu/DOVE/blob/main/config/bird/test_bird_toy.yml.

The code might expect a mask image and a bounding box file for each image, but they are not really used during inference. So, you could create dummy (all white) images for the masks. For the *_box.txt, you could simply write (assuming the crops are resized to 256:

0, 0, 0, 256, 256, 256, 256, 0

which stand for global_frame_id, crop_x0, crop_y0, crop_w, crop_h, full_w, full_h, sharpness, as documented here.