#921·unilm

[unimim] mismatched positional_embed about vit-large/14 for input resolution with 196

Author: futureisatyourhandCreated Nov 17, 2022Updated Apr 4, 2026

hello, for CLIP knowledge distilation paper, i.e.,A Unified View of Masked Image Modeling: when the teacher is CLIP vit-large/14 for 196's input resolution, and the student is vit-base/16 for 224's input resolution, vit-large/14's positional embed (i.e.,257) for CLIP mismatch with the positional embed of our teacher (i.e., 197). How should I fix this to align with the paper.

Thanks very much!