(Feature Request) Model EMA

In my fine-tuning experiments, I've run into catastrophic forgetting and was wondering if using EMA would help mitigate this. I'm not sure if it makes sense to do this to the image encoder alone, or both the text + image encoder.

If it makes sense, I'd love to try implementing this with some guidance.