[BUG] A plain success() output guardrail silently discards an earlier guardrail's successWith() rewrite

Author: moyu-byCreated Sep 16, 2026Updated Sep 16, 2026

LabelsGuardrails

Summary

In an output guardrail chain, a successWith(...) rewrite is silently discarded if any later guardrail in the same chain returns a plain success(). The caller then receives the original, un-rewritten model response — while every guardrail still reports success, with no warning, log, or exception.

Four minimal cases, all with the same stub model returning ORIGINAL:

Chain	Actual return
`[Rewrite, PlainSuccess]`	`ORIGINAL` :x:
`[PlainSuccess, Rewrite]`	`REWRITTEN` :white_check_mark:
`[Rewrite]` (single)	`REWRITTEN` :white_check_mark:
`[PlainSuccess, Rewrite, PlainSuccess]`	`ORIGINAL` :x:

So the effective rule is "only the last guardrail's returned text survives", which is not documented anywhere. It makes any redaction / masking / format-fixing guardrail a silent no-op unless it happens to be declared last.

Environment

LangChain4j 1.20.0 (langchain4j-core, langchain4j)
JDK 25
No provider involved — reproduced with a stub ChatModel, zero network

Minimal reproduction

java

package com.moyu.servicedesk.guardrail;

import dev.langchain4j.data.message.AiMessage;
import dev.langchain4j.guardrail.OutputGuardrail;
import dev.langchain4j.guardrail.OutputGuardrailResult;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.UserMessage;
import org.junit.jupiter.api.Test;

public class GuardrailChainProbe {

    public interface Assistant {
        String chat(@UserMessage String message);
    }

    static ChatModel stubModelReturningOriginal() {
        return new ChatModel() {
            @Override
            public ChatResponse chat(ChatRequest chatRequest) {
                return ChatResponse.builder()
                        .aiMessage(AiMessage.from("ORIGINAL"))
                        .build();
            }
        };
    }

    static class RewriteGuardrail implements OutputGuardrail {
        @Override
        public OutputGuardrailResult validate(AiMessage responseFromLLM) {
            return successWith("REWRITTEN");
        }
    }

    static class PlainSuccessGuardrail implements OutputGuardrail {
        @Override
        public OutputGuardrailResult validate(AiMessage responseFromLLM) {
            return success();
        }
    }

    private static void probe(String label, OutputGuardrail... guardrails) {
        Assistant assistant = AiServices.builder(Assistant.class)
                .chatModel(stubModelReturningOriginal())
                .outputGuardrails(guardrails)
                .build();

        System.out.println("[PROBE] " + label + " -> " + assistant.chat("x"));
    }

    @Test
    void probeAllFourChainConfigurations() {
        probe("(a) [Rewrite, PlainSuccess]              ", new RewriteGuardrail(), new PlainSuccessGuardrail());
        probe("(b) [PlainSuccess, Rewrite]              ", new PlainSuccessGuardrail(), new RewriteGuardrail());
        probe("(c) [Rewrite]    (only one)              ", new RewriteGuardrail());
        probe("(d) [PlainSuccess, Rewrite, PlainSuccess]", new PlainSuccessGuardrail(), new RewriteGuardrail(), new PlainSuccessGuardrail());
    }
}

Actual output:

[PROBE] (a) [Rewrite, PlainSuccess]              -> ORIGINAL
[PROBE] (b) [PlainSuccess, Rewrite]              -> REWRITTEN
[PROBE] (c) [Rewrite]    (only one)              -> REWRITTEN
[PROBE] (d) [PlainSuccess, Rewrite, PlainSuccess] -> ORIGINAL
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Root cause

AbstractGuardrailExecutor#executeGuardrails keeps two separate accumulators:

java

var accumulatedRequest = request;
var accumulatedResult = createSuccess();

for (var guardrail : this.guardrails) {
    var result = validate(accumulatedRequest, guardrail);

    if (result.isFatal()) {
        return handleFatalResult(accumulatedResult, result);
    }
    if (result.hasRewrittenResult()) {
        accumulatedRequest = accumulatedRequest.withText(result.successfulText());  // rewrite lives here
    }
    accumulatedResult = composeResult(accumulatedResult, result);                   // text does not
}
return accumulatedResult;

The rewrite is carried forward on the request channel (so later guardrails do see the rewritten text), but the result channel — which is what the method returns — is replaced on every iteration:

java

protected R composeResult(R oldResult, R newResult) {
    if (oldResult.isSuccess()) {
        return newResult;          // <-- always taken on an all-success chain
    }
    if (newResult.isSuccess()) {
        return oldResult;
    }
    // failures are merged here; this branch is only reached when both failed
    var failures = new ArrayList<F>(oldResult.failures());
    failures.addAll(newResult.failures());
    return createFailure(failures);
}

isSuccess() cannot distinguish the two kinds of success:

java

default boolean isSuccess() {
    var result = result();
    return (result == Result.SUCCESS) || (result == Result.SUCCESS_WITH_RESULT);
}

Since SUCCESS_WITH_RESULT counts as success, on a chain that never fails oldResult.isSuccess() is always true, so composeResult degenerates to "take the last result" and the failure-merging logic never runs on this path.

The rewrite only reaches the caller if the final result carries it — see OutputGuardrailResult#createResponse:

java

private ChatResponse createResponse(OutputGuardrailRequest params) {
    var response = params.responseFromLLM();
    return response.toBuilder()
            .aiMessage(hasRewrittenResult() ? successfulAiMessage : response.aiMessage())
            .build();
}

OutputGuardrailExecutor#rewriteResult looks like it should rescue this case, but it cannot on the non-reprompt path:

java

public OutputGuardrailResult execute(OutputGuardrailRequest request) {
    var accumulatedRequest = request;
    ...
    result = rewriteResult(request, accumulatedRequest, executeGuardrails(accumulatedRequest));
    ...
    if (++attempt < maxAttempts) {
        accumulatedRequest = OutputGuardrailRequest.builder()   // reassigned ONLY on the reprompt branch
                .responseFromLLM(response) ... .build();
    }
}

java

private OutputGuardrailResult rewriteResult(
        OutputGuardrailRequest originalRequest,
        OutputGuardrailRequest validatedRequest,
        OutputGuardrailResult result) {
    if (result.isSuccess() && !result.hasRewrittenResult()) {
        String originalText = originalRequest.responseFromLLM().aiMessage().text();
        String validatedText = validatedRequest.responseFromLLM().aiMessage().text();
        if (!originalText.equals(validatedText)) {
            return successWith(originalRequest.responseFromLLM().aiMessage().withText(validatedText));
        }
    }
    return result;
}

On a passing first attempt originalRequest == validatedRequest (same object), so the texts are always equal and this branch is never taken. Additionally, the accumulatedRequest mutated inside executeGuardrails is a method-local variable — executeGuardrails returns R, not the request, so the rewritten request never escapes.

Why this looks like a bug rather than intended behaviour

success() at position n means "the input I received is acceptable". The input a later guardrail receives is already the rewritten text (the rewrite is applied to accumulatedRequest before that guardrail runs). So reverting to the original text contradicts the semantics of success() at that position: it should approve the text it actually saw, not discard a decision made earlier in the chain.

Suggested fix

In AbstractGuardrailExecutor#composeResult, distinguish the two success kinds:

java

if (oldResult.isSuccess() && newResult.isSuccess()) {
    return newResult.hasRewrittenResult() ? newResult : oldResult;
}

Workaround

Declare every successWith guardrail last in the chain. This ordering constraint is not mentioned in the guardrails documentation, which only advises ordering guardrails by cost / failure frequency.

#3820 ("Fix guardrails result after a successful reprompt", released in 1.8.0) fixed a neighbouring symptom of this same result-selection logic — that precedent suggests result-selection mistakes here are treated as bugs.
Not the same as #4496: that report is about ChatMemory storing the raw response despite a rewrite. This report is about the value returned to the caller; no memory is involved in the reproduction above.
Related in spirit to #5728 and #4316 (rewrite handling on the streaming path), but this one reproduces on the synchronous, non-streaming path with no provider.

Impact

Silent and hard to notice: guardrails execute and report success, no warning or exception is raised, and the caller receives un-rewritten content. Any output guardrail used for PII redaction, format enforcement, or content replacement is a no-op unless it is declared last.

Source: langchain4j/langchain4j

View original on GitHub View discussion on GitHub

[BUG] A plain success() output guardrail silently discards an earlier guardrail's successWith() rewrite

Summary

Environment

Minimal reproduction

Root cause

Why this looks like a bug rather than intended behaviour

Suggested fix

Workaround

Related

Impact