Baike.dev
All toolsAI codingTrendingOpen sourceNewsSubmit
Log in
Back to tool/Back to issues
#3333·text-generation-inference

How to use prefix caching

Author: Noha-MagdyCreated Sep 27, 2025Updated Sep 29, 2025

Hi I can't find a way to turn on the prefix caching

When I run any model, I always get: Using prefix caching = False

Thanks a lot

Source: huggingface/text-generation-inference

View original on GitHubView discussion on GitHub