#475·open_clip

Speed up model.generate() with coca?

Author: PclanglaisCreated Mar 26, 2023Updated Apr 18, 2023

I am building an image classification workflow on top of coca captions and embeddings. The only downside is that this is slow (about 100/images per minute on a google colab).

So two related questions:

  • Is it possible to extract the embeddings calculated within model.generate()? Currently I use encode_image on top which is basically a duplicate.
  • Are there some settings that may speed up model.generate at the expense of accuracy? In my current workflow I only need the top characteristic words from the captions of images that belong to the same cluster. I'm not entirely clear how beamsearch work.