memory-efficient attention is default opened? if i dont use flash attn
Author: wac81Created Apr 7, 2023Updated Apr 24, 2023
or if i want use memory-efficient attention, i must call scaled_dot_product_attention?
PyTorch 2.0 includes an optimized and memory-efficient attention implementation through the torch.nn.functional.scaled_dot_product_attention function
Source: lucidrains/PaLM-rlhf-pytorch