otlpexporter (gRPC) does not use the ProfilesDictionary for Resource attributes
Component(s)
pdata/pprofile, exporter/otlp
What happened?
Describe the bug
#15794 (not yet merged) changes otlphttpexporter to marshal profiles via pprofileotlp.ExportRequest.MarshalProto(), which delegates to pprofile.ProtoMarshaler.MarshalProfiles. That method calls convertProfilesToReferences, interning resource/scope attribute strings into the ProfilesDictionary string table before marshaling.
otlpexporter (gRPC) does not go through this path: its client calls rawClient.Export(ctx, request.orig, ...) (pdata/pprofile/pprofileotlp/grpc.go), passing the raw *internal.ExportProfilesServiceRequest struct directly rather than the ExportRequest wrapper.
The gRPC codec (pdata/internal/otelgrpc/encoding.go) then calls SizeProto()/MarshalProto() straight on that raw struct, bypassing pprofile.ProtoMarshaler and convertProfilesToReferences entirely.
As a result, profiles sent over gRPC never get resource/scope attribute strings interned into the dictionary.
Steps to reproduce
package pprofileotlp
import (
"context"
"net"
"sync"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/resolver"
"google.golang.org/grpc/test/bufconn"
"go.opentelemetry.io/collector/pdata/pprofile"
)
func TestExportInternAttributeStrings(t *testing.T) {
lis := bufconn.Listen(1024 * 1024)
s := grpc.NewServer()
var received ExportRequest
RegisterGRPCServer(s, &capturingProfilesServer{received: &received})
wg := sync.WaitGroup{}
wg.Go(func() {
assert.NoError(t, s.Serve(lis))
})
t.Cleanup(func() {
s.Stop()
wg.Wait()
})
resolver.SetDefaultScheme("passthrough")
cc, err := grpc.NewClient("bufnet",
grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) {
return lis.Dial()
}),
grpc.WithTransportCredentials(insecure.NewCredentials()))
require.NoError(t, err)
t.Cleanup(func() {
assert.NoError(t, cc.Close())
})
pd := pprofile.NewProfiles()
pd.ResourceProfiles().AppendEmpty().Resource().Attributes().PutStr("service.name", "my-service")
_, err = NewGRPCClient(cc).Export(context.Background(), NewExportRequestFromProfiles(pd))
require.NoError(t, err)
// "service.name" and "my-service" should be interned in the dictionary.
assert.Contains(t, received.Profiles().Dictionary().StringTable().AsRaw(), "service.name")
assert.Contains(t, received.Profiles().Dictionary().StringTable().AsRaw(), "my-service")
}
type capturingProfilesServer struct {
UnimplementedGRPCServer
received *ExportRequest
}
func (f capturingProfilesServer) Export(_ context.Context, request ExportRequest) (ExportResponse, error) {
*f.received = request
return NewExportResponse(), nil
}What did you expect to see? Key and value from Resource attribute should have been interned ("service.name" and "my-service"),
What did you see instead? An empty string table.
Collector version
v0.159.0
Environment information
Environment
OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
Log output
Additional context
pdata/xpdata/request.MarshalProfiles (used by exporterhelper's persistent sending-queue encoding for profiles) also bypasses convertProfilesToReferences.
That path round-trips entirely on local disk and is immediately unmarshaled back by the same code, so the only potential gain there is a smaller on-disk queue footprint, not reduced network egress.
Flagging it in case persistent-queue disk usage for profiles turns out to matter.
Tip
React with to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.
Source: open-telemetry/opentelemetry-collector