cache=True reuses artifacts across different NUMBA_SLP_VECTORIZE / NUMBA_OPT settings

Reporting a bug

• I have tried using the latest released version of Numba (0.65.1).
• I have included a self-contained code sample to reproduce the problem.

Summary

The on-disk cache key does not include most of the configuration that drives code generation. A function compiled and cached with NUMBA_SLP_VECTORIZE=1 is silently reused by a later process running with NUMBA_SLP_VECTORIZE=0, and vice versa: the run executes machine code built under a configuration it did not ask for, with no warning. The same holds for NUMBA_OPT, NUMBA_LOOP_VECTORIZE and NUMBA_DEBUGINFO (measured below); NUMBA_ENABLE_AVX is unaffected, because it already reaches the key through the CPU feature string.

I hit this while benchmarking a matrix-free finite-element kernel: enabling NUMBA_SLP_VECTORIZE is worth 1.65x on that kernel, and benchmark results stopped being reproducible until I noticed that __pycache__ was serving artifacts from earlier runs with the other setting.

Reproducer

Self-contained, no third-party packages beyond numpy.

import json, os, pathlib, pickle, sys, time

os.environ["NUMBA_SLP_VECTORIZE"] = sys.argv[1] if len(sys.argv) > 1 else "0"

import numba
import numpy as np
from numba import njit


@njit(cache=True, fastmath=True, boundscheck=False, error_model="numpy")
def kernel(x, out, A, C):
    """Short fixed-length loops with local scratch: the shape SLP vectorises."""
    for e in range(x.shape[0]):
        u = np.empty(24); f = np.zeros(24); a = np.empty(6); b = np.empty(6)
        for i in range(24):
            u[i] = x[e, i]
        for g in range(8):
            for k in range(6):
                acc = 0.0
                for i in range(24):
                    acc += A[g, k, i] * u[i]
                a[k] = acc
            for k in range(6):
                acc = 0.0
                for j in range(6):
                    acc += C[k, j] * a[j]
                b[k] = acc
            for k in range(6):
                s = b[k]
                for i in range(24):
                    f[i] += A[g, k, i] * s
        for i in range(24):
            out[e, i] = f[i]


def index_entries():
    files = sorted(pathlib.Path("__pycache__").glob("*.nbi"))
    if not files:
        return 0
    with open(files[0], "rb") as handle:
        pickle.load(handle)                       # version stamp
        _, overloads = pickle.loads(handle.read())
    return len(overloads)


rng = np.random.default_rng(0)
A = np.ascontiguousarray(rng.standard_normal((8, 6, 24)))
C = np.ascontiguousarray(rng.standard_normal((6, 6)))
x = np.ascontiguousarray(rng.standard_normal((200_000, 24)))
out = np.empty_like(x)

started = time.perf_counter()
kernel(x, out, A, C)
first_call = time.perf_counter() - started
times = []
for _ in range(5):
    moment = time.perf_counter()
    kernel(x, out, A, C)
    times.append(time.perf_counter() - moment)

print(json.dumps({
    "NUMBA_SLP_VECTORIZE": os.environ["NUMBA_SLP_VECTORIZE"],
    "first_call_s": round(first_call, 3),
    "steady_ms": round(1000 * float(np.median(times)), 1),
    "cache_hits": sum(kernel.stats.cache_hits.values()),
    "cache_misses": sum(kernel.stats.cache_misses.values()),
    "entries_in_nbi_index": index_entries(),
}))

Run it three times:

rm -rf __pycache__
python repro.py 1     # (1) compiles with the flag on
python repro.py 0     # (2) flag off, but the cache answers
rm -rf __pycache__
python repro.py 0     # (3) what run (2) should have been

Observed

(1) {"NUMBA_SLP_VECTORIZE": "1", "first_call_s": 0.626, "steady_ms": 98.9,
     "cache_hits": 0, "cache_misses": 1, "entries_in_nbi_index": 1}
(2) {"NUMBA_SLP_VECTORIZE": "0", "first_call_s": 0.364, "steady_ms": 102.8,
     "cache_hits": 1, "cache_misses": 0, "entries_in_nbi_index": 1}
(3) {"NUMBA_SLP_VECTORIZE": "0", "first_call_s": 0.653, "steady_ms": 146.1,
     "cache_hits": 0, "cache_misses": 1, "entries_in_nbi_index": 1}

Run (2) is a cache hit and runs at the flag-on speed (102.8 ms); run (3), on a clean cache with the same environment, compiles and runs at 146.1 ms. The index never holds more than one entry, so the two configurations share a key.

Reversing the order reproduces the mirror image: a flag-off artifact is served to a flag-on process, which then silently loses the vectorisation it asked for.

Why

numba/core/caching.py, Cache._index_key:

return (sig, codegen.magic_tuple(), (hasher(codebytes), hasher(cvarbytes)))

numba/core/codegen.py, CPUCodegen.magic_tuple:

def magic_tuple(self):
    """
    Return a tuple unambiguously describing the codegen behaviour.
    """
    return (self._llvm_module.triple, self._get_host_cpu_name(),
            self._tm_features)

The tuple carries target triple, CPU name and target-machine features, but none of the settings that drive the optimisation pipeline.

Note that this is not a case of "configuration was deliberately left out of the key": two config values are already in there. _get_host_cpu_name returns config.CPU_NAME when it is set, and _get_host_cpu_features returns config.CPU_FEATURES when it is set; NUMBA_ENABLE_AVX also reaches the key indirectly, because get_host_cpu_features strips the avx* features from the string that ends up in the tuple. So the design already treats codegen-affecting configuration as part of the cache identity - the pipeline options are simply the ones that fell through.

I checked which ones, by running the same cached function twice over one cache directory with two different values (script attached as check_which_options_key.py):

NUMBA_OPT and NUMBA_DEBUGINFO are the ones I should expect to bite people most often: both are ordinary debugging knobs. Turning debug info on and being served a cached artifact without it looks like "numba's debug info is broken" rather than like a cache problem.

Impact

• A cached function can execute code compiled under settings the current process did not request - silently, in both directions.
• Benchmarks and performance regressions become irreproducible until __pycache__ is cleared; the symptom looks like machine noise.
• It is hard to notice, because inspect_asm() is disabled for cached code (Inspection disabled for cached code), so the usual way to check what was generated is unavailable exactly when this bites.

Suggested fix

Extend magic_tuple with the settings that affect code generation, so the existing key mechanism separates them. Locally I run this monkey patch, which makes the two configurations occupy two entries in the same .nbi index:

from numba.core import config
from numba.core.codegen import CPUCodegen

CODEGEN_OPTIONS = ("OPT", "LOOP_VECTORIZE", "SLP_VECTORIZE", "DEBUGINFO_DEFAULT")

_original = CPUCodegen.magic_tuple


def magic_tuple(self):
    options = tuple(f"{name}={getattr(config, name, None)!s}"
                    for name in CODEGEN_OPTIONS)
    return (*_original(self), options)


CPUCodegen.magic_tuple = magic_tuple

With the patch, run (2) above becomes a cache miss and produces 146 ms, and repeated runs of either configuration hit their own entry. magic_tuple is called in exactly one place in the codebase - Cache._index_key - so the blast radius is limited to the cache key.

(ENABLE_AVX is deliberately absent from that list: it already reaches the key through the CPU feature string, as measured above.)

The list is what my check found, not an audit of the pass pipeline; maintainers will know the authoritative set - NUMBA_DEBUG*, NUMBA_DUMP_* and anything else that alters the emitted module would belong there on the same grounds.

numba/cuda/codegen.py defines its own magic_tuple and would need the same treatment if the CUDA target has equivalent settings; I did not look.

Environment

• numba 0.65.1, llvmlite 0.46.0 (pip)
• numpy 2.4.4, Python 3.14.5
• Windows 11, x86-64 (Intel Core Ultra 9 285H)