ti.ad.Tape(validation=True) misses loop-carried read-after-write, returning a silently wrong gradient

Describe the bug

The global data access rule checker catches a write-after-read within one loop iteration, but not the loop-carried form (read f[i], write f[i+1]). The tape then returns a gradient ~4.8x the true one, with exact forward values and no warning.

To Reproduce

import math, taichi as ti
ti.init(arch=ti.cpu, default_fp=ti.f64, debug=True)

E, SIG_Y, N = 1000.0, 2.0, 60
th   = ti.field(ti.f64, shape=(), needs_grad=True)
acc  = ti.field(ti.f64, shape=(), needs_grad=True)
epsp = ti.field(ti.f64, shape=(N + 1,), needs_grad=True)

@ti.kernel
def plasticity():                      # reads epsp[i], writes epsp[i+1]
    ti.loop_config(serialize=True)
    for i in range(N):
        trial = E * (th[None] * ti.sin(2.0 * math.pi * i / N) - epsp[i])
        d = 0.0
        if ti.abs(trial) > SIG_Y:
            d = (ti.abs(trial) - SIG_Y) / E
            acc[None] += d
        epsp[i + 1] = epsp[i] + (d if trial > 0 else -d)

th[None] = 0.0045
with ti.ad.Tape(acc, validation=True):     # <- raises nothing
    plasticity()
print("value", acc[None], "  grad", th.grad[None], "  true grad 3.895472")

This is a 1-D perfect-plasticity return map. 3.895472 is the derivative of an independent float64 implementation of the same model, confirmed by central differences.

Log

[Taichi] version 1.7.4, llvm 15.0.4, commit b4b956fd, linux, python 3.10.12
[Taichi] Starting on arch=x64
value 0.007529621915295555   grad 18.843885969656736   true grad 3.895472

The forward value matches the independent implementation to eight digits; only the derivative is wrong. Across theta in [0.0045, 0.0065] the ratio stays between 4.68x and 4.94x.

Additional comments

The checker does work here — controls, so this is not a report about validation silently not running:

Writing the same model the documented way — one kernel per step, so no field is both read and written inside a single kernel — gives the correct gradient. So this is an encoding that breaks the rule rather than a limitation of autodiff on the model; the request is only that the checker say so.

Diagnosis, offered tentatively. The checker looks like it tests whether an element is overwritten. Here every element of epsp is written once and read once, one iteration apart, so a per-element test has nothing to fire on while the dependence the rule forbids is still there. That is consistent with all three cases below and with #2425: everything it catches reads and writes the same element. We have not read the checker's implementation, so this is a hypothesis, not a claim.

What would help. Either of:

1. detect the loop-carried case; or
2. warn whenever a needs_grad field is both read and written in a serialised loop, even without proving a violation — a false positive is far cheaper here than a silent 4.8x gradient error.

We recognise the checker is marked experimental (#5719), so if neither is feasible in the near term, then at minimum it would help to document that it does not cover loop-carried dependences — so users know what a passing validation=True does and does not mean. That seems like a floor rather than a resolution, though: the failure is silent and the forward values are exact, so a user has no signal that anything is wrong.

The checker already closed the previously-reported case. #2425 (open since 2021) reports a silently wrong gradient from an exponentially-weighted moving average, ewma[row] = 0.5*data[row,col] + 0.5*ewma[row], which @k-ye diagnosed in-thread as the global data access rule. On 1.7.4 the checker catches that kernel. So this report is not "the checker does not work" — it plainly does, and it covers the case that prompted it. The gap is narrower than that: the caught case reads and writes the same element, and the case here reads epsp[i] and writes epsp[i+1], so no element is ever overwritten and a per-element test has nothing to fire on.

If that framing is right, then this is the natural next case for the same checker rather than a separate problem, and #2425 could probably be closed once it is covered.

Related. #5476 / #5541 / #5570 added the checker, #5719 moved it to experimental, #6347 updated the docs, #2425 is the earlier report the checker now catches. No existing issue appears to cover the different-index case.