Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#48·PaLM-rlhf-pytorch

memory-efficient attention is default opened? if i dont use flash attn

Author: wac81Created Apr 7, 2023Updated Apr 24, 2023

or if i want use memory-efficient attention, i must call scaled_dot_product_attention?

PyTorch 2.0 includes an optimized and memory-efficient attention implementation through the torch.nn.functional.scaled_dot_product_attention function

Source: lucidrains/PaLM-rlhf-pytorch

View original on GitHubView discussion on GitHub