Durable agents silently skip scorers when observability is configured

Author: rphansen91Created Sep 17, 2026Updated Sep 17, 2026

LabelsEvalsObservability (AI Telemetry)effort:lowimpact:mediumstatus: auto-triaged

Summary

When an agent with configured scorers is wrapped in createDurableAgent, scorers silently never execute if observability is configured. No warning, no error — scoring just stops.

The same agent, unwrapped, scores fine with the identical Mastra config. And the durable wrap also scores fine when observability is absent — which is likely why tests don't catch this.

Case	Observability	Scorer runs
plain `Agent`	off	✅
`createDurableAgent`	off	✅
plain `Agent`	on	✅
`createDurableAgent`	on	❌ never

Verified on @mastra/[email protected] + @mastra/[email protected].

Suspected mechanism

In the durable agentic workflow, the map-final-output step ends the agent/model observability spans (observability.rebuildSpan(agentSpanData)?.end(...)), and only then does the execute-scorers step run executeDurableAgentScorers with that same tracing context.

runScorer has an early return:

javascript

const currentSpan = observabilityContext.tracing?.currentSpan;
if (currentSpan?.isValid === false) return;

Since the span was already ended in the prior step, the scorer bails silently. With no observability configured, currentSpan is undefined and scoring proceeds — matching the table above.

Real-world impact

We run a production diagnosis agent with a quality scorer at sampling rate 1.0. The day we wrapped it in createDurableAgent, scoring stopped dead (last score written minutes before the deploy) with zero log evidence. Our deployment uses MastraPlatformExporter, so any observability-enabled durable agent loses all scoring.

Repro

Self-contained, no API keys (uses MockLanguageModelV3):

javascript

// Repro: do agent-configured scorers run when the agent is wrapped in
// createDurableAgent? Core 1.63.0. Compares non-durable vs durable stream.
import { Mastra } from '@mastra/core/mastra';
import { Agent } from '@mastra/core/agent';
import { createDurableAgent } from '@mastra/core/agent/durable';
import { InMemoryStore } from '@mastra/core/storage';
import { createScorer } from '@mastra/core/evals';
import { Observability, DefaultExporter } from '@mastra/observability';
import { MockLanguageModelV3, simulateReadableStream } from 'ai/test';

const model = new MockLanguageModelV3({
  doStream: async () => ({
    stream: simulateReadableStream({
      chunks: [
        { type: 'text-start', id: '1' },
        { type: 'text-delta', id: '1', delta: 'hello' },
        { type: 'text-end', id: '1' },
        { type: 'finish', finishReason: 'stop', usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 } },
      ],
    }),
  }),
});

let scorerRuns = 0;
const scorer = createScorer({
  id: 'repro-scorer',
  name: 'Repro Scorer',
  description: 'counts runs',
}).generateScore(() => {
  scorerRuns += 1;
  return 1;
});

async function runCase(label, wrap) {
  scorerRuns = 0;
  const agent = new Agent({
    id: 'a1',
    name: 'a1',
    instructions: 'test',
    model,
    scorers: { reproScorer: { scorer, sampling: { type: 'ratio', rate: 1 } } },
  });
  const registered = wrap ? createDurableAgent({ agent, maxSteps: 5 }) : agent;
  const storage = new InMemoryStore({ id: 's1' });
  const mastra = new Mastra({
    agents: { a1: registered },
    scorers: { reproScorer: scorer },
    storage,
    observability: new Observability({ configs: { default: { serviceName: 'repro', exporters: [new DefaultExporter()] } } }),
    logger: false,
  });
  const a = mastra.getAgent('a1');
  const stream = await a.stream('hi');
  for await (const _ of stream.fullStream) { /* drain */ }
  await new Promise((r) => setTimeout(r, 3000)); // scorers are fire-and-forget
  let saved = 'n/a';
  try {
    const store = await storage.getStore('scores');
    const res = await store.getScoresByEntityId?.({ entityId: 'a1', entityType: 'AGENT', pagination: { page: 0, perPage: 10 } });
    saved = res?.scores?.length ?? res?.pagination?.total ?? JSON.stringify(res)?.slice(0, 100);
  } catch (e) {
    saved = `err: ${e.message}`;
  }
  console.log(`${label}: scorer executed ${scorerRuns}x, saved scores: ${saved}`);
}

await runCase('non-durable', false);
await runCase('durable   ', true);
process.exit(0);

Output:

non-durable: scorer executed 1x
durable   : scorer executed 0x

Remove the observability: line from the Mastra config and both cases score.

Expected

Durable agents should run configured scorers regardless of span lifecycle — or at minimum log a warning when scoring is skipped, instead of silently dropping it.

Source: mastra-ai/mastra

View original on GitHub View discussion on GitHub