About the training of SR_1024 and the differentia between SR_256 and SR_1024?

Author: BIGJUN777Created Jul 26, 2022Updated Jun 21, 2024

Hi there, I have some confusing problems below:

  1. As mentioned in the paper, there is a 200M of parameters difference between SR_256 and SR_1024. image However, based on the architecture specification listed in Sec. F.2 and Sec. F.3 in the paper, these two models are almost the same, except for using the self-attention layer or not. Does anyone know how this 200M difference comes?

  2. About the training of SR_1024, does anyone know the meaning of "Our 256x256->1024x1024 super-resolution model trains on 64x64->256x256 crops of the 1024x1024 image"? Why does this relate to the self-attention layer? image

Source: lucidrains/imagen-pytorch