How to use prefix caching
Author: Noha-MagdyCreated Sep 27, 2025Updated Sep 29, 2025
Hi I can't find a way to turn on the prefix caching
When I run any model, I always get: Using prefix caching = False
Thanks a lot
Source: huggingface/text-generation-inference