#4789·baml

[bug] Nightly TypeScript bridge Collector stays empty while ai.ModelTurn reports usage

Author: BenSpexCreated Sep 9, 2026Updated Sep 9, 2026

Environment

@boundaryml/baml-bridge and installed toolchain: 0.18.1-nightly.20260828.a
Node v24.11.1, Linux 6.8.0-139-generic, x64
Local deterministic Chat Completions stub; no real provider, credentials, private data or paid requests required.

Expected behavior

A Collector passed to callFunction(rt, name, kwargs, undefined, [collector], callContext) should receive the direct LLM call and a language wrapper’s nested call, including usage and timings. The Collector documentation describes these outputs. If host collectors are intentionally unsupported in this nightly runtime, please document that and consider an explicit error instead of silent empty results.

Reproduction

With the matching toolchain installed, install @boundaryml/[email protected], save the following as repro.mjs, and run node repro.mjs with Node 24:

javascript

import http from "node:http";
import * as bridge from "@boundaryml/baml-bridge";
const server = http.createServer((req, res) => {
  req.resume();
  req.on("end", () => {
    res.writeHead(200, { "content-type": "application/json" });
    res.end(JSON.stringify({
      id: "fake", object: "chat.completion", created: 1, model: "fake-model",
      choices: [{ index: 0, message: { role: "assistant", content: JSON.stringify({ value: "ok" }) }, finish_reason: "stop" }],
      usage: { prompt_tokens: 11, completion_tokens: 7, total_tokens: 18, prompt_tokens_details: { cached_tokens: 3 } }
    }));
  });
});
await new Promise(resolve => server.listen(0, "127.0.0.1", resolve));
const port = server.address().port;
const source = `
class Result { value string }
class JournalWire { value string input int? output int? cached int? calls int }
function Direct() -> Result {
  client: openai.ChatClient.new(model="fake-model", api_key="dummy", base_url="http://127.0.0.1:${port}/v1", capture_wire=true)
  prompt: \`${'${role("user")}'} Return ok. ${'${ctx.output_format()}'}\`
}
function Wrapped() -> Result { Direct() }
function ViaJournal() -> JournalWire {
  let spec = Direct$spec();
  let turn = openai.ChatClient.new(model="fake-model", api_key="dummy", base_url="http://127.0.0.1:${port}/v1", capture_wire=true).invoke(
    ai.ModelTurnInput { prompt: spec.prompt_template, journal: ai.Journal.new(spec), toolbox: spec.toolbox, output_type: spec.output_type() }
  );
  let value = baml.sap.parse<Result>(turn.terminal_text() ?? "");
  JournalWire { value: value.value, input: turn.usage?.input_tokens, output: turn.usage?.output_tokens, cached: turn.usage?.cached_input_tokens, calls: turn.calls.length() }
}`;
try {
  const runtime = bridge.BamlRuntime.initializeRuntime("/virtual", { "main.baml": source });
  for (const name of ["Direct", "Wrapped"]) {
    const collector = new bridge.Collector(name);
    const out = (await bridge.callFunction(runtime, name, {}, undefined, [collector], new bridge.BamlCallContext())).result();
    const u = collector.usage;
    console.log(JSON.stringify({ name, out, usage: { inputTokens: u.inputTokens, outputTokens: u.outputTokens, cachedInputTokens: u.cachedInputTokens }, logs: collector.logs.length, calls: collector.logs.flatMap(x => x.calls).length }));
  }
  console.log(JSON.stringify({ name: "ViaJournal", out: (await bridge.callFunction(runtime, "ViaJournal", {}, undefined, undefined, new bridge.BamlCallContext())).result() }));
} finally { server.close(); }

Actual output

json

{"name":"Direct","out":{"value":"ok"},"usage":{"inputTokens":null,"outputTokens":null,"cachedInputTokens":null},"logs":0,"calls":0}
{"name":"Wrapped","out":{"value":"ok"},"usage":{"inputTokens":null,"outputTokens":null,"cachedInputTokens":null},"logs":0,"calls":0}
{"name":"ViaJournal","out":{"value":"ok","input":11,"output":7,"cached":3,"calls":1}}

All three requests succeed and parse the valid JSON. ai.ModelTurn sees exactly the usage supplied by the stub, proving that the provider response parser has the data. Both direct and nested host-collected calls leave the Collector empty.

Additional checks and impact

The installed bridge signature/order was checked in its implementation and declarations; it maps the supplied collectors through _native().
This reproduces without streaming, retries, generic reflected schemas, or model-generated malformed JSON.
The application impact is missing token/cost/latency telemetry despite successful model calls. We keep unknown usage as null and currently work around this using ai.Journal/ai.ModelTurn usage and the selected call’s timing.
I searched existing Collector/usage issues and did not find this direct/nested-call nightly reproduction.

Source: BoundaryML/baml

View original on GitHub View discussion on GitHub