Baike.dev
Log in
Back to tool/Back to issues
#4845·TensorRT

TensorRT 10.14+ multi-GPU build regression with Cask-backed ops

Author: braindevicesCreated Sep 5, 2026Updated Sep 5, 2026
LabelsModule:Engine Build

Summary

Starting with TensorRT 10.14.1, IBuilder::buildSerializedNetwork() can fail when building a second engine after switching to a different CUDA device in the same process.

The failure is specific to Cask-backed operators; the minimal repro uses MaxPool:

Error Code: 9: Skipping tactic ... Cask Pooling Runner Execute Failure
Error Code: 10: Internal Error (Could not find any implementation for node MaxPool_0.
In computeCosts at .../optimizer.cpp:4265)

The same-device sequence succeeds. Building either device alone succeeds. Building each engine in a separate process succeeds.

The regression is present in 10.14.1.48.post1 through 10.16.1.11 and is not present in 10.13.3.9 or 11.2.1.2.

Environment

  • TensorRT: tensorrt-cu13

    • 10.14.1.48.post1, 10.15.1.29, 10.16.0.72, 10.16.1.11: broken
    • 10.13.2.6, 10.13.3.9: OK
    • 11.2.1.2: OK

  • GPUs:

    • Quadro RTX 4000, SM 7.5
    • NVIDIA RTX 4000 SFF Ada Generation, SM 8.9

  • Driver: 580.142

  • OS: AlmaLinux 9.8, kernel 6.12.103-1.el9.x86_64

  • Python: 3.11

Minimal reproducer

Requires two CUDA devices.

#!/usr/bin/env python3
import ctypes
import sys

import onnx
import tensorrt as trt
from onnx import TensorProto, helper


def make_model() -> bytes:
    x = helper.make_tensor_value_info("x", TensorProto.FLOAT, [1, 1, 8, 8])
    node = helper.make_node(
        "MaxPool", ["x"], ["y"], name="MaxPool_0",
        kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1],
    )
    y = helper.make_tensor_value_info("y", TensorProto.FLOAT, [1, 1, 8, 8])
    graph = helper.make_graph([node], "tiny_maxpool", [x], [y])
    model = helper.make_model(graph, opset_imports=[helper.make_opsetid("", 20)])
    model.ir_version = 9
    onnx.checker.check_model(model)
    return model.SerializeToString()


def build(model: bytes, cudart: ctypes.CDLL, device: int, label: str) -> bool:
    assert cudart.cudaSetDevice(device) == 0
    print(f"======={label} on {device=}", file=sys.stderr)
    logger = trt.Logger(trt.Logger.INFO)
    builder = trt.Builder(logger)
    config = builder.create_builder_config()
    config.set_memory_pool_limit(
        trt.MemoryPoolType.WORKSPACE, 1 << 30
    )

    network = builder.create_network(
        1 << int(trt.NetworkDefinitionCreationFlag.STRONGLY_TYPED)
    )
    parser = trt.OnnxParser(network, logger)
    assert parser.parse(model, path="/tmp")
    engine = builder.build_serialized_network(network, config)
    print("=======Failed" if engine is None else "Succeed", file=sys.stderr)


def main():
    print(f"======={trt.__version__=}", file=sys.stderr)
    cudart = ctypes.CDLL("libcudart.so")
    count = ctypes.c_int()
    cudart.cudaGetDeviceCount(ctypes.byref(count))
    print(f"======={count.value=}", file=sys.stderr)
    if count.value < 2:
        return 1

    model = make_model()

    build(model, cudart, 0, "same device 1/2")
    build(model, cudart, 0, "same device 2/2") # succeeds

    build(model, cudart, 0, "different device 1/2")
    build(model, cudart, 1, "different device 2/2") # fails after device switch

    return 0

if __name__ == "__main__":
    main()

Full traceback / output (TensorRT 10.16.1.11):

=======trt.__version__='10.16.1.11'
=======count.value=2
=======same device 1/2 on device=0
[09/05/2026-19:32:08] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 47, GPU 99 (MiB)
[09/05/2026-19:32:08] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:08] [TRT] [I] Input filename:   /tmp
[09/05/2026-19:32:08] [TRT] [I] ONNX IR version:  0.0.9
[09/05/2026-19:32:08] [TRT] [I] Opset version:    20
[09/05/2026-19:32:08] [TRT] [I] Producer name:    
[09/05/2026-19:32:08] [TRT] [I] Producer version: 
[09/05/2026-19:32:08] [TRT] [I] Domain:           
[09/05/2026-19:32:08] [TRT] [I] Model version:    0
[09/05/2026-19:32:08] [TRT] [I] Doc string:       
[09/05/2026-19:32:08] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:08] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[09/05/2026-19:32:08] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +91, GPU +2, now: CPU 335, GPU 101 (MiB)
[09/05/2026-19:32:08] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[09/05/2026-19:32:08] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[09/05/2026-19:32:09] [TRT] [I] Detected 1 inputs and 1 output network tensors.
[09/05/2026-19:32:09] [TRT] [I] Total Host Persistent Memory: 4160 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Device Persistent Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Max Scratch Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Activation Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Weights Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Engine generation completed in 0.412562 seconds.
[09/05/2026-19:32:09] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 1 MiB
Succeed
=======same device 2/2 on device=0
[09/05/2026-19:32:09] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:09] [TRT] [I] Input filename:   /tmp
[09/05/2026-19:32:09] [TRT] [I] ONNX IR version:  0.0.9
[09/05/2026-19:32:09] [TRT] [I] Opset version:    20
[09/05/2026-19:32:09] [TRT] [I] Producer name:    
[09/05/2026-19:32:09] [TRT] [I] Producer version: 
[09/05/2026-19:32:09] [TRT] [I] Domain:           
[09/05/2026-19:32:09] [TRT] [I] Model version:    0
[09/05/2026-19:32:09] [TRT] [I] Doc string:       
[09/05/2026-19:32:09] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:09] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[09/05/2026-19:32:09] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU -89, GPU +2, now: CPU 383, GPU 103 (MiB)
[09/05/2026-19:32:09] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[09/05/2026-19:32:09] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[09/05/2026-19:32:09] [TRT] [I] Detected 1 inputs and 1 output network tensors.
[09/05/2026-19:32:09] [TRT] [I] Total Host Persistent Memory: 4160 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Device Persistent Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Max Scratch Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Activation Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Weights Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Engine generation completed in 0.171036 seconds.
[09/05/2026-19:32:09] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 1 MiB
Succeed
=======different device 1/2 on device=0
[09/05/2026-19:32:09] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:09] [TRT] [I] Input filename:   /tmp
[09/05/2026-19:32:09] [TRT] [I] ONNX IR version:  0.0.9
[09/05/2026-19:32:09] [TRT] [I] Opset version:    20
[09/05/2026-19:32:09] [TRT] [I] Producer name:    
[09/05/2026-19:32:09] [TRT] [I] Producer version: 
[09/05/2026-19:32:09] [TRT] [I] Domain:           
[09/05/2026-19:32:09] [TRT] [I] Model version:    0
[09/05/2026-19:32:09] [TRT] [I] Doc string:       
[09/05/2026-19:32:09] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:09] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[09/05/2026-19:32:09] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU -89, GPU +2, now: CPU 383, GPU 103 (MiB)
[09/05/2026-19:32:09] [TRT] [I] BuilderFlag::kTF32 is set but hardware does not support TF32. Disabling TF32.
[09/05/2026-19:32:09] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[09/05/2026-19:32:09] [TRT] [I] Detected 1 inputs and 1 output network tensors.
[09/05/2026-19:32:09] [TRT] [I] Total Host Persistent Memory: 4160 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Device Persistent Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Max Scratch Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Activation Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Total Weights Memory: 0 bytes
[09/05/2026-19:32:09] [TRT] [I] Engine generation completed in 0.174508 seconds.
[09/05/2026-19:32:09] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 1 MiB
Succeed
=======different device 2/2 on device=1
[09/05/2026-19:32:09] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:09] [TRT] [I] Input filename:   /tmp
[09/05/2026-19:32:09] [TRT] [I] ONNX IR version:  0.0.9
[09/05/2026-19:32:09] [TRT] [I] Opset version:    20
[09/05/2026-19:32:09] [TRT] [I] Producer name:    
[09/05/2026-19:32:09] [TRT] [I] Producer version: 
[09/05/2026-19:32:09] [TRT] [I] Domain:           
[09/05/2026-19:32:09] [TRT] [I] Model version:    0
[09/05/2026-19:32:09] [TRT] [I] Doc string:       
[09/05/2026-19:32:09] [TRT] [I] ----------------------------------------------------------------
[09/05/2026-19:32:09] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU -58, GPU +0, now: CPU 413, GPU 169 (MiB)
[09/05/2026-19:32:09] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xca723f3a6d5c572a due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x1b13dea2697d92fc due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x5f085e81bccfdfd4 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xadf1751d9d52bc14 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x4805490eb56c6142 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xd4b9db5d6f22d997 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x4fc018d7ee59fb45 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x39a96deeea00c898 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x5154ad0527677a6f due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x80cf0fe785959355 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xe8f436d3bcc38289 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x31efd02f5d31b9ba due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x6abffa4236113b26 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x5faf4a0a8a5670ed due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x62f9478381204a04 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xbc27f0a63be7eb30 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xb59f9cfb90407c92 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x96dbfe2b0fabc076 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xc5cd6987d3ba33c8 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x9e9d43eab89ab154 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x9e04eb8087eb1de8 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x6cfdc01ca6767e28 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x1022263884083147 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x7213e6ffae85b15b due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x388b14a64cc134ea due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xbbe2a17f60d27137 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x6193a97a80c05353 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xf987a1a777e5f6d6 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xc17f7d05d2cb7b56 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x8cfd5f7bc7e409c2 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xbf50b5fd61a339a9 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xcd8bd446648b42ea due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x18649bf933394065 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x664b53018051029a due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xef31670ae7f6188e due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xf660830175fd03a3 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xc6ba2cdc89fee151 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x1cd68f7b324808fb due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x0b7e8a3b56789516 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x2bc9717f957e700e due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x618e02d8db64989d due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xb33a917a9f2c32a9 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x78dfe6d3496f83b0 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xb716083cd692488b due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xdfebc8d71bf5fa7c due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xe74c45c0759b786b due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x8382d5c464539e87 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xaec8628e8180bced due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xd76bac5638836f8a due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xfa211b1cdd504de0 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xe9d01a2a900075cb due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x789b2859f2e03e79 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0xbd3963b8ccd084c6 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic 0x2c7251cbae30cf74 due to exception Cask Pooling Runner Execute Failure In execute at /_src/runtime/gpu/cask/poolingRunner.cpp:324
[09/05/2026-19:32:09] [TRT] [E] Error Code: 9: Skipping tactic

Source: NVIDIA/TensorRT

View original on GitHubView discussion on GitHub