#118·minGPT

Output of CausalSelfAttention

Author: whchan05Created Jul 6, 2023Updated May 6, 2024

It seems that the output of this block is simply reshaped from multiple heads. From the original "Attention is all you need" paper, it seems that there is another linear layer, W^O. Mind if I ask if this is intentional or an error? Thank you