cp-vton-plus

CP-VTON+ (CVPRW 2020)

Official implementation for "CP-VTON+: Clothing Shape and Texture Preserving Image-Based Virtual Try-On" from CVPRW 2020.

Project page: https://minar09.github.io/cpvtonplus/.

Saved/Pre-trained models: Checkpoints

Dataset: VITON_PLUS

The code and pre-trained models are tested with pytorch 0.4.1, torchvision 0.2.1, opencv-python 4.1 and pillow 5.4 (Python 3 env).

Project page | Paper | Dataset | Model | Video

Usage

This pipeline combines consecutive training and testing of GMM and TOM. GMM generates the warped clothes according to the target human. Then, TOM blends the warped clothes outputs from GMM into the target human properties to generate the final try-on output.

1. Install the requirements
2. Download/Prepare the dataset
3. Train GMM network
4. Get warped clothes for the training set with a trained GMM network, and copy warped clothes & masks inside the data/train directory
5. Train TOM network
6. Test GMM for the testing set
7. Get warped clothes for the testing set, copy warped clothes & masks inside the data/test directory
8. Test TOM testing set

Installation

This implementation is built and tested in PyTorch 0.4.1. Pytorch and torchvision are recommended to install with conda: conda install pytorch=0.4.1 torchvision=0.2.1 -c pytorch

For all packages, run pip install -r requirements.txt

Data Preparation

For training/testing VITON dataset, our full and processed dataset is available here: https://1drv.ms/u/s!Ai8t8GAHdzVUiQRFmTPrtrAy0ZP5?e=rS1aK8. After downloading, unzip to your data directory.

Training

Run python train.py with your specific usage options for the GMM and TOM stage.

For example, GMM: python train.py --name GMM --stage GMM --workers 4 --save_count 5000 --shuffle

Then run test.py for GMM network with the training dataset, which will generate the warped clothes and masks in the "warp-cloth" and "warp-mask" folders inside the "result/GMM/train/" directory. Copy the "warp-cloth" and "warp-mask" folders into your data directory, for example inside the "data/train" folder.

Run TOM stage, python train.py --name TOM --stage TOM --workers 4 --save_count 5000 --shuffle

Testing

Run 'python test.py' with your specific usage options.

For example, GMM: python test.py --name GMM --stage GMM --workers 4 --datamode test --data_list test_pairs.txt --checkpoint checkpoints/GMM/gmm_final.pth

Then run test.py for GMM network with the testing dataset, which will generate the warped clothes and masks in the "warp-cloth" and "warp-mask" folders inside the "result/GMM/test/" directory. Copy the "warp-cloth" and "warp-mask" folders into your data directory, for example inside the "data/test" folder.

Run TOM stage: python test.py --name TOM --stage TOM --workers 4 --datamode test --data_list test_pairs.txt --checkpoint checkpoints/TOM/tom_final.pth

Inference/Demo

Download the pre-trained models from here: https://1drv.ms/u/s!Ai8t8GAHdzVUiQA-o3C7cnrfGN6O?e=EaRiFP. Then run the same step as Testing to test/infer our model. The code and pre-trained models are tested with PyTorch 0.4.1, torchvision 0.2.1, opencv 4.1 and pillow 5.4.

Testing with custom images

to run the model with custom internet images, make sure you have the following:

1. image (image of a person, crop/resize to 192 x 256 (width x height) pixels)
2. image-parse (you can generate with CIHP_PGN or Graphonomy pretrained networks from the person image. See this comment)
3. cloth (in-shop cloth image, crop/resize to 192 x 256 (width x height) pixels)
4. cloth-mask (binary mask of cloth image, you can generate it with simple pillow/opencv function)
5. pose (pose keypoints of the person, generate with openpose COCO-18 model (OpenPose from the official repository is preferred))
6. Also, make a test_pairs.txt file for your custom images. Follow the VITON dataset format to keep same arrangements, otherwise you can modify the code.

What to do in case of unexpected results

There are many factors that can cause distorted/unexpected results. Can you please do the following?

1. First try the original viton dataset and test pair combinations, check the intermediate results and the final output. Check if they are as expected.
2. If the original viton results are not as expected, please check the issues raised in this GitHub repo, people have already found several issues, and see how they solved them.
3. If the original viton test results are as expected, then run your custom test sets and check the intermediate results and debug where its going wrong.
4. If you are testing with custom images then check the GitHub repository readme and related issues on how to run with custom images.

Its difficult to understand your issue from only single image/output. As I mentioned, there are various factors. Please debug yourself step by step and see where its going wrong. Check all the available intermediate/final inputs/outputs visually, and check multiple cases to see if the issue is happening for all cases. Good luck to you!

Citation

Please cite our paper in your publications if it helps your research:

@InProceedings{Minar_CPP_2020_CVPR_Workshops,
	title={CP-VTON+: Clothing Shape and Texture Preserving Image-Based Virtual Try-On},
	author={Minar, Matiur Rahman and Thai Thanh Tuan and Ahn, Heejune and Rosin, Paul and Lai, Yu-Kun},
	booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
	month = {June},
	year = {2020}
}

Acknowledgements

This implementation is largely based on the PyTorch implementation of CP-VTON. We are extremely grateful for their public implementation.