Clarification on Pose to Body
I have a few questions regarding the pose --> body task. From the paper,
Dance video dataset. We download YouTube dance videos for the pose to human motion synthesis task. Each video is about 3 ∼ 4 minutes at 1280 × 720 resolution, and we crop the central 512×720 regions. We extract human poses with the DensePose [21] and the OpenPose [5] algorithms, and directly concatenate the results together. The training set includes a dance video from a single dancer, while the test set contains videos of other dance motions or from other dancers.
By "directly concatenate" do you also layer the DensePose UV pose on the OpenPose color-coded pose? Or just use one of the two, and if so which performs better?
The provided example shows a still background for the dancer. If I understand correctly would a video with changing background or multiple people cause issues? e.g. the demonstration for DensePose and OpenPose includes a fast paced multi-person dance video. However the poses generated are all more or less synchronized, and the background is not encoded in any way. Would the model generate mostly noise for the background in this case, and would it be able to synthesize human bodies from multiple pose estimations?
How does the test set perform against other dancers? Do changes such as height, limb length, etc. cause issues in generation?
Lastly, is there a dockerized version of this repository available? Alternatively, would this model compile inside the Flownet2 Docker container?
Thanks!
Source: NVIDIA/vid2vid