question about rotary positional encoding (rope) for text.
Author: yashkantCreated Aug 9, 2024Updated Jan 5, 2026
hi authors, thanks a lot for releasing the code!
i noticed that position ids for text embeddings is set to zeros at this line: https://github.com/black-forest-labs/flux/blob/c23ae247225daba30fbd56058d247cc1b1fc20a3/src/flux/sampling.py#L51
i am wondering why this is the case, esp. when compared to the image position ids being a 2D meshgrid here: https://github.com/black-forest-labs/flux/blob/c23ae247225daba30fbd56058d247cc1b1fc20a3/src/flux/sampling.py#L42
is it because t5 model already adds positional encoding to the text, perhaps?
thanks!
Source: black-forest-labs/flux