Recommended approach for OTel initiated from HTTP API to gRPC backend services

In what area(s)?

/area runtime

/area operator

/area placement

/area docs

/area test-and-release

/area injector

/area scheduler

/area sentry

Ask your question here

I have a set of microservices running .NET 10 using Dapr 1.17.3-msft.1. I have a Web API out front fielding HTTP calls directly exposed out through an application gateway ingress controller. From the API I make calls to my backend services using proxy service invocation by using Grpc.Net.Client against the sidecar port. When I do this, I get 2 traceparent headers concatenated together in the backend service, which makes OTel freak out and create a new traceparent. This only started happening when we went from Dapr 1.15.9-msft.1 to 1.17. Setting the grpc-trace-bin header directly also seems to make dapr freak out and headers start streaming async (and I don't always end up with them all) so there seems to be some difference in behavior there. Is there a way to have dapr figure out that it isn't starting the distributed trace chain or a different approach I could be taking?