#1935·Guardrails

bug: streaming output rails reuse stale `$bot_message` / `$user_message` action params across chunks

Author: fallintoplaceCreated May 26, 2026Updated Sep 11, 2026

Summary

When rails.output.streaming.enabled=True, output-rail actions that receive substituted params like text=$bot_message or bot_response=$bot_message can reuse stale values across streaming chunks and even across later requests on the same LLMRails instance.

This looks like a shared-mutation bug in the streaming output-rails path, not a general streaming failure.

Scope

This seems to affect only the combination of:

  • stream_async()
  • output rails configured
  • rails.output.streaming.enabled=True
  • output-rail actions that rely on direct substituted kwargs from $bot_message / $user_message

Actions that read context["bot_message"] instead of the substituted kwargs appear to be unaffected.

Why this happens

In the streaming path:

  • get_action_details_from_flow_id() returns element["action_params"] directly
  • _prepare_params() then mutates that dict in place while replacing $bot_message / $user_message

Relevant spots:

  • nemoguardrails/rails/llm/utils.py returns the original action_params object from the parsed flow config
  • nemoguardrails/rails/llm/llmrails.py mutates action_params in _prepare_params() before executing the rail action

Because the dict comes straight from the parsed flow config, the first processed chunk can permanently replace the placeholder value. Later chunks then reuse the first chunk's text instead of the current chunk, and later requests on the same LLMRails instance can also inherit the stale value.

Minimal shape of the problem

A flow like this is enough to trigger it:

colang
define flow capture output
  execute capture_output(text=$bot_message)

If capture_output records the text kwarg during streaming and the stream yields multiple chunks like A, B, C:

Expected:

  • first call sees A
  • second call sees B
  • third call sees C

Actual:

  • first call sees A
  • later calls can keep seeing A

The same stale value can also survive into a later streamed request on the same LLMRails instance.

Affected built-in rails

This does not look limited to custom flows. There are built-in output rails that pass $bot_message directly into action params, for example:

  • prompt_security (protect response)
  • privateai output PII detection/masking
  • gliner output PII detection/masking
  • policyai output moderation
  • trend_micro output guard
  • activefence output moderation
  • ai_defense response inspection
  • regex output check

Test coverage gap

The existing streaming output-rails tests seem to use actions that read context["bot_message"], which is probably why this path is not being caught today.

Possible fix

Copy action_params before placeholder substitution, for example in _prepare_params() and/or when returning from get_action_details_from_flow_id().

Something along the lines of:

python
action_params = dict(action_params or {})

would avoid mutating the parsed flow config object in place.