Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#60·PaLM-rlhf-pytorch

A bug in the implementation of the top-p sampling

Author: allblueJTCreated Sep 27, 2024Updated Sep 27, 2024

https://github.com/lucidrains/PaLM-rlhf-pytorch/blob/6b02ee329106baff78e293afa7d1d2e6dd4e5ca2/palm_rlhf_pytorch/utils.py#L60

Using the sorted indices to index the sorted indices does not make sense. I think it may be return logits.scatter(1, sorted_indices, sorted_logits)

Source: lucidrains/PaLM-rlhf-pytorch

View original on GitHubView discussion on GitHub