rewrite_tag: splitting a log group across tags drops group metadata
Bug
When rewrite_tag sends records from one log group to different tags, only the first record's tag gets the group metadata and attributes. Records on other tags lose them.
Reproduces with v5.1.2 (official image, linux/amd64). rewrite_tag.c is unchanged on master.
Reproduction
An info log and a payment-failure log share the resource attributes service.name=checkout and deployment.environment.name=production. Retag them by level and print the exported OTLP JSON:
service:
flush: 1
grace: 1
log_level: error
pipeline:
inputs:
- name: dummy
tag: source
dummy: '{"level":"info","message":"order accepted"} {"level":"error","message":"payment authorization failed"}'
samples: 1
fixed_timestamp: true
start_time_sec: 1700000000
processors:
logs:
- name: opentelemetry_envelope
- name: content_modifier
context: otel_resource_attributes
action: upsert
key: service.name
value: checkout
- name: content_modifier
context: otel_resource_attributes
action: upsert
key: deployment.environment.name
value: production
filters:
- name: rewrite_tag
match: source
rule: $level ^(info|error)$ routed.$level false
outputs:
- name: stdout
match: routed.*
format: otlp_json
workers: 0fluent-bit -c repro.yamlObserved
Both messages are exported, but the payment-failure record has an empty resource:
| Exported message | service.name |
deployment.environment.name |
|---|---|---|
order accepted |
checkout |
production |
payment authorization failed |
missing | missing |
A query filtering on those service and environment attributes would miss the failure record, even though it was delivered.
OTLP JSON output{
"resourceLogs": [
{
"resource": {
"attributes": [
{
"key": "service.name",
"value": {
"stringValue": "checkout"
}
},
{
"key": "deployment.environment.name",
"value": {
"stringValue": "production"
}
}
]
},
"scopeLogs": [
{
"scope": {},
"logRecords": [
{
"timeUnixNano": "1700000000000000000",
"body": {
"stringValue": "order accepted"
}
}
]
}
]
}
]
}
{
"resourceLogs": [
{
"resource": {},
"scopeLogs": [
{
"scope": {},
"logRecords": [
{
"timeUnixNano": "1700000000000000000",
"body": {
"stringValue": "payment authorization failed"
}
}
]
}
]
}
]
}Retagging both records to routed.same preserves both records' resource attributes. Reversing the input order moves the loss to order accepted.
Expected
Both exported records retain service.name=checkout and deployment.environment.name=production, regardless of their destination tags.
Cause
cb_rewrite_tag_filter() passes process_record() the raw bytes since the previous record (source). Only the first slice contains the group opener, so later records emitted under another tag arrive without one.
Source: fluent/fluent-bit