response-transformer logs spurious "body transform failed" warnings on large/chunked JSON responses
Is there an existing issue for this?
- I have searched the existing issues
Kong version
Reproduced by code inspection against master @ 391ee48.
Current Behavior
kong.response.get_raw_body() is documented and implemented to return nil on every body_filter invocation that is not the last chunk of the response body (it buffers internally via ngx.ctx.KONG_BODY_BUFFER and only returns the full body once eof is reached):
-- @treturn string body The full body when the last chunk has been read,
-- otherwise returns `nil`.
-- @usage
-- local body = kong.response.get_raw_body()
-- if body then
-- body = transform(body)
-- kong.response.set_raw_body(body)
-- endOther first-party plugins consuming this function correctly guard against nil, e.g. kong/plugins/proxy-cache/handler.lua:
local body = kong.response.get_raw_body()
if body then
...
endkong/plugins/response-transformer/handler.lua#body_filter does not:
function ResponseTransformerHandler:body_filter(conf)
if not is_body_transform_set(conf) or not is_json_body(...) then
return
end
local body = kong.response.get_raw_body()
local json_body, err = transform_json_body(conf, body)
if err then
kong.log.warn("body transform failed: " .. err)
return
end
return kong.response.set_raw_body(json_body)
endOn every intermediate body_filter call (i.e. any JSON response whose body doesn't fit in a single nginx output buffer — large or chunk-transferred responses), body is nil. transform_json_body(conf, nil) -> body_transformer.parse_json(nil) returns nil, so transform_json_body returns nil, "failed parsing json body", and the handler logs:
kong.log.warn("body transform failed: failed parsing json body")This happens once per intermediate buffer, for completely normal, successful traffic — exactly the scenario already exercised (without asserting on log output) by spec/03-plugins/15-response-transformer/05-big_response_body_spec.lua (1MB JSON body). The final chunk is still transformed correctly since get_raw_body() returns the full body on eof, so there's no functional/response-correctness impact — but at volume this floods logs and can mislead operators debugging genuine transformation failures or trigger log-based alerting false positives.
Expected Behavior
body_filter should not attempt to transform (or log a warning about) a nil body on intermediate chunks; it should only act once the full body is available.
Steps To Reproduce
Configure the response-transformer plugin on a route with config.add.json or config.remove.json set, and proxy a request whose response is JSON and large enough to span multiple nginx output buffers (e.g. ~1MB, as in the existing big-response-body test). Observe [warn] ... body transform failed: failed parsing json body logged once per intermediate buffer while the final response body is still transformed correctly.
Anything else?
I have a fix ready and will open a pull request referencing this issue.
Source: Kong/kong