OTLP/JSON silently drops every batch containing a float attribute: serde_json arbitrary_precision breaks the AnyValue visitor

Bug Description

OTLP/JSON ingestion rejects any request containing an AnyValue with a doubleValue, returning HTTP 400 and discarding the entire batch. Logs and traces are both affected. Metrics are not.

The cause is the workspace enabling serde_json's arbitrary_precision feature in Cargo.toml:

serde_json = { version = "1", features = ["arbitrary_precision"] }

With arbitrary_precision, serde_json represents every JSON number as an internal map rather than a primitive. opentelemetry-proto's hand-written AnyValue visitor (deserialize_from_value in opentelemetry-proto/src/proto.rs) reads that branch as a primitive:

"doubleValue" => { let d = map.next_value()?; ... }   // expects f64, receives a map

The result is invalid type: map, expected f64, raised at src/api/ingest/src/request/logs/ingest.rs where the body is deserialized with serde_json::from_slice::<ExportLogsServiceRequest>.

Only doubleValue is affected. stringValue and boolValue are not numbers. intValue survives because it deserializes through StringOrInt, an untagged enum that tolerates the map form. doubleValue is the sole AnyValue branch reading a bare JSON number.

Metrics are unaffected because src/core/src/metrics/otlp_json_compat.rs normalizes metric payloads before deserialization, and asDouble datapoints do not route through this visitor. src/core/src/profiles/otlp_json_compat.rs provides the same protection for profiles. Logs and traces have no equivalent shim.

The failure is not partial. A well-formed record batched alongside a float-bearing record is discarded with it, because the rejection happens during deserialization of the whole ExportLogsServiceRequest. This is silent from the producer's side: OTLP exporters do not surface receiver 400s to application code, so the data is simply absent.

This is reachable from ordinary traffic. Any log or span attribute carrying a non-integer value — a ratio, a percentage, a fractional duration — triggers it. In a local DevContainer running the OpenTelemetry JS SDK against OpenObserve v0.92.2, a 60-second soak produced 201 rejected batches, and 20 idle requests produced one.

Root cause isolation

The opentelemetry-proto crate is not at fault. Isolating the variable, with opentelemetry-proto = "=0.32.0" and no other change:

Column 242 matches the error returned by the running server byte-for-byte for the same payload.

Related but distinct: open-telemetry/opentelemetry-rust#3677 covers the same visitor rejecting null AnyValue fields, fixed by #3682, merged 2026-09-07 and not yet published. That fix wraps each branch in Option<T> and addresses null handling only; it does not make the visitor arbitrary_precision-safe, so upgrading opentelemetry-proto will not resolve this.

Steps to Reproduce

Against any OpenObserve instance:

curl -u "$USER:$PASS" -H 'Content-Type: application/json' \
  -d '{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"p"}}]},"scopeLogs":[{"logRecords":[{"timeUnixNano":"1789000000000000000","body":{"stringValue":"x"},"attributes":[{"key":"r","value":{"doubleValue":1.5}}]}]}]}]}' \
  "$BASE/api/$ORG/v1/logs"

Returns:

{"code":400,"message":"Invalid json: invalid type: map, expected f64 at line 1 column 242"}

Replacing {"doubleValue":1.5} with {"stringValue":"s"} returns 200. The same payload against /v1/traces with a span attribute fails identically.

The root cause reproduces without an OpenObserve build, in about twenty lines:

[dependencies]
opentelemetry-proto = { version = "=0.32.0", default-features = false, features = ["gen-tonic-messages", "with-serde", "logs"] }
serde_json = { version = "1", features = ["arbitrary_precision"] }

use opentelemetry_proto::tonic::common::v1::AnyValue;

fn main() {
    println!("{:?}", serde_json::from_str::<AnyValue>(r#"{"doubleValue":1.5}"#));
}

Removing features = ["arbitrary_precision"] changes the result from Err to Ok(AnyValue { value: Some(DoubleValue(1.5)) }).

Affected positions

Probed individually against v0.92.2:

Possible fixes

Listed for discussion; I have not assumed which you would prefer.

• Drop arbitrary_precision, if the precision guarantee is not load-bearing. This is the smallest change but the widest blast radius, since the feature is workspace-wide through Cargo feature unification and something may depend on it.
• Scope arbitrary_precision to the crates that need it rather than enabling it workspace-wide.
• Add a logs/traces OTLP JSON compat shim mirroring metrics/otlp_json_compat.rs, normalizing doubleValue before deserialization. Consistent with the existing pattern, and localized.
• Upstream an arbitrary_precision-tolerant AnyValue visitor to opentelemetry-proto, which would remove the need for a third shim. Slower, and depends on their release cadence.

Happy to prepare a PR for whichever direction you prefer. Please indicate before I start, since the options differ substantially in scope.

Note that src/core/src/logs/otlp.rs already contains a DoubleValue(1.23) test that passes while this bug is live: it constructs the AnyValue struct directly and calls handle_request, exercising the protobuf path rather than JSON deserialization. Any fix should add coverage at the deserialization layer, or it will repeat the same blind spot.

Environment Details

• OpenObserve v0.92.2, single-node, official Docker image
• opentelemetry-proto 0.32.0 per Cargo.lock at the v0.92.2 tag
• Producer: OpenTelemetry JS SDK, @opentelemetry/exporter-logs-otlp-http and -trace-otlp-http, which emit OTLP/HTTP JSON
• Root cause isolated independently with Rust 1.98.1 on Linux x86_64

Is this a regression?

Not established. I have not tested earlier releases. The arbitrary_precision feature and the AnyValue visitor would both need version-by-version checking to date the regression, and I have not done that.

Priority and severity

Offered as an observation rather than a judgment on your triage. The data loss is silent and reaches ordinary application traffic, and the blast radius is the whole batch rather than the offending record. Against that, it requires OTLP/JSON specifically — the OpenTelemetry Collector defaults to protobuf — which likely explains why it appears not to have been reported before.

Affected functionalities

Ingestion — logs and traces via OTLP/JSON.

Investigation and reproduction produced with Claude Code.