About Classifier-free Guidance for Two Conditionings
Author: ardianumamCreated Feb 28, 2025Updated May 29, 2025
Hi,
Thanks for the awesome work! I have read the paper and still not sure about several particular aspects, specifically regarding the reconstructed target for each conditional input case. I write my guesses below, and please help to confirm.
- Only the input image ($c_I$) is set to a null value for 5% of the examples. In this case, the model is trained to use only the instruction text, so, the reconstructed image target will be the edited image.
- Only the text instruction ($c_T$) is set to a null value for 5% of the examples. Here, the model focuses to learn the denoising by only having the image condition (associated with the original image), so, the reconstructed image target will be the original image.
- Both $c_I$ and $c_T$ are set to null values for another 5% of the examples. This situation trains the model without any condition, so, the reconstructed image target will be the original image.
- Both conditionings are used (remaining 85% of the examples), therefore, the model is trained to denoise into target edited image.
Thanks much and appreciate!
Source: timothybrooks/instruct-pix2pix