是否使用 trtllm-build 而不是 optimum-nvidia 来构建引擎,或者是使用了不正确的 optimum-nvidia 版本?

Hello, I'm experiencing significant issues when trying to use Text Generation Inference (TGI) with TensorRT-LLM as the backend. Problem 1: Version Compatibility I cannot use the latest version of TGI due to a known bug (see: https://GitHub.com/huggingface/text-generation-inference/issues/3296). I'm therefore using version: ghcr.io/huggingface/text-generation-inference:3.3.4-trtllm However, this version uses TensorRT-LLM v0.17.0.post1, while the latest optimum-nvidia version ([v0.1.0b9]) uses TensorRT-LLM 0.16.0. When I try to launch TGI with my engine built using optimum-nvidia, I get the following error: ``` root@5ddf177112d7:/usr/local/tgi/bin# /usr/local/tgi/bin/text-generation-launcher --model-id "/engines/LLaMA-3.2-3b-instruct-optimum/GPU/engines" --tokenizer-name "/models/LLaMA-3.2-3b-instruct" --executor-worker "/usr/local/tgi/bin/executorWorker" 2025-07-27T06:16:40.717109Z INFO text_generation_backends_trtllm: backends/trtllm/src/main.rs:293: Successfully retrieved tokenizer /models/LLaMA-3.2-3b-instruct [2025-07-27 06:16:40.717] [info] [ffi.hpp:164] Initializing TGI - TensoRT-LLM Backend (v0.17.0.post1) [2025-07-27 06:16:40.747] [info] [ffi.hpp:173] [FFI] Detected 1 Nvidia GPU(s) [2025-07-27 06:16:40.758] [info] [backend.cpp:22] Detected single engine deployment, using leader mode [TensorRT-LLM][INFO] Engine version 0.16.0 found in the config file, assuming engine(s) built by new builder API. [TensorRT-LLM][INFO] Initializing MPI with thread mode 3 [TensorRT-LLM][INFO] Initialized MPI [TensorRT-LLM][INFO] Refreshed the MPI local session [TensorRT-LLM][INFO] MPI size: 1, MPI local size: 1, rank: 0 [TensorRT-LLM][INFO] Rank 0 is using GPU 0 [TensorRT-LLM][INFO] TRTGptModel maxNumSequences: 64 [TensorRT-LLM][INFO] TRTGptModel maxBatchSize: 64 [TensorRT-LLM][INFO] TRTGptModel maxBeamWidth: 1 [TensorRT-LLM][INFO] TRTGptModel maxSequenceLen: 4096 [TensorRT-LLM][INFO] TRTGptModel maxDraftLen: 0 [TensorRT-LLM][INFO] TRTGptModel mMaxAttentionWindowSize: (4096) * 28 [TensorRT-LLM][INFO] TRTGptModel enableTrtOverlap: 0 [TensorRT-LLM][INFO] TRTGptModel normalizeLogProbs: 1 [TensorRT-LLM][INFO] TRTGptModel maxNumTokens: 262144 [TensorRT-LLM][INFO] TRTGptModel maxInputLen: 4095 = maxSequenceLen - 1 since chunked context is enabled [TensorRT-LLM][INFO] If model type is encoder, maxInputLen would be reset in trtEncoderModel to maxInputLen: 4096 = maxSequenceLen. [TensorRT-LLM][INFO] Capacity Scheduler Policy: MAX_UTILIZATION [TensorRT-LLM][INFO] Context Chunking Scheduler Policy: None [TensorRT-LLM][INFO] Loaded engine size: 6981 MiB [TensorRT-LLM][ERROR] IRuntime::deserializeCudaEngine: Error Code 6: API Usage Error (The engine plan file is not compatible with this version of TensorRT, expecting library version 10.8.0.43 got ..) Error: Runtime("[TensorRT-LLM][ERROR] Assertion failed: Failed to deserialize cuda engine. …