Docker Image that works on RTX 5090
Author: shawnrushefskyCreated May 9, 2025Updated Dec 9, 2025
Feature request
Please release a docker image built for cuda 12.8+ that will work on rtx 5090.
Motivation
I work at Salad. Our customers want to run LLMs on RTX 5090. We like to recommend TGI because it’s so simple to set up, but that is not currently true for the latest GPUs.
Your contribution
Last I checked, PyTorch nightly was still needed for these GPUs
Source: huggingface/text-generation-inference