Question about Classifier-free Guidance
Author: lucalaCreated Apr 15, 2023Updated Jan 8, 2024
When embedding my text for conditioning, the trick for classifier-free guidance is to drop the embedding sometimes (usually 10% of the time).
My question is, what does drop mean? It seems I have come across two variants: using a random tensor as a substitute or a zero tensor.
GLIDE mentions in section 2.3 "we sometimes replace text captions with an empty sequence" - this would be a third option, using the embedding from the empty string?
I haven't been able to find any explanation on this, does someone know?
Source: lucidrains/imagen-pytorch