A bug in the implementation of the top-p sampling
Author: allblueJTCreated Sep 27, 2024Updated Sep 27, 2024
Using the sorted indices to index the sorted indices does not make sense.
I think it may be
return logits.scatter(1, sorted_indices, sorted_logits)
Source: lucidrains/PaLM-rlhf-pytorch