OpenTelemetry shutdown runs with an already-cancelled context on SIGTERM, dropping the final export
Author: roannyCreated Sep 17, 2026Updated Sep 17, 2026
Summary
On a graceful shutdown (SIGTERM, for example a Cloud Run scale-down), the server runs the OpenTelemetry shutdown with a context that the signal handler has already cancelled. The result is an ERROR-level log line on every shutdown:
error shutting down OpenTelemetry: context canceled
context canceledThe line is more than noise. The final export is abandoned: the last metrics collection is never exported, and spans still queued race the process exit.
Version: v1.11.0, run with --telemetry-gcp. The same code is still on main at 7ed7f55.
Where it happens (v1.11.0)
cmd/root.go:403createsctx, cancel := context.WithCancel(cmd.Context()). The goroutine at:409-424callscancel()when SIGTERM or SIGINT arrives.cmd/root.go:426passes thatctxtoopts.Setup(ctx). At:430-432the telemetry shutdown is deferred asdefer func() { _ = shutdown(ctx) }(), using the same context.- The HTTP server is treated differently. At
cmd/root.go:524-528it gets a freshcontext.WithTimeout(context.Background(), 10*time.Second). The telemetry shutdown gets no equivalent. internal/telemetry/telemetry.go:SetupOTel's shutdown iserrors.Join(tracerProvider.Shutdown(ctx), meterProvider.Shutdown(ctx)). That is why the message holds twocontext canceled, one per provider.cmd/internal/options.go:126-131logs the joined error at ERROR.
Why the final export is lost (go.opentelemetry.io/otel/sdk v1.44.0, as pinned in go.mod)
- Metrics.
PeriodicReader.Shutdown(sdk/metric/periodic_reader.go:368) derives its timeout from the parent context. Here the parent is already cancelled, so the derived context is cancelled too. It then callscollect, which reachespipeline.produce(sdk/metric/pipeline.go:131).producereturnsctx.Err()before collecting anything (:136). The final collection is therefore never exported. This path is deterministic. - Traces.
batchSpanProcessor.Shutdown(sdk/trace/batch_span_processor.go:161-189) starts the drain in a goroutine, then selects onwaitagainstctx.Done(). The context is already done, so it returnsctx.Err()at once without waiting.run()then returns,mainreturns, and the process exits. Queued spans are exported only if the drain wins that race.
Suggested fix
Give the telemetry shutdown its own bounded context, as run() already does for the HTTP server. context.WithoutCancel keeps the logger and the other values carried by ctx. The package already uses it in cmd/internal/options.go.
defer func() {
shutdownCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer cancel()
_ = shutdown(shutdownCtx)
}()What we verified, and what we did not
- Verified by reading the code at v1.11.0 and at the OpenTelemetry SDK version it pins: every reference above.
- Observed in a Cloud Run deployment: the log line appears on instance shutdowns and accounts for most ERROR-severity entries of an otherwise healthy service.
- Not measured: that specific spans or metric points from an instance's final interval are missing in Cloud Trace and Cloud Monitoring. The conclusion that they are lost rests on the SDK code paths above.
Source: googleapis/mcp-toolbox