[bug] Playground Tests panel shows "No tests found" since 0.19.0: requestCollectTests returns an empty tree
Product
VS Code Playground
Describe the bug
Since toolchain 0.19.0 the playground's Tests panel is empty for every project. It shows "No tests found", while baml test --list in the same project lists the tests and baml test runs them.
The server answers requestCollectTests with an empty tree (testCollectionResult data [], no expandError). The build is current (isBexCurrent: true), the $init_test_<file> function is present in the function list, and the server log shows no error. The same project on 0.18.1-nightly.20260909.a returns the expected tree.
| Toolchain | baml test --list |
testCollectionResult |
|---|---|---|
| 0.18.0 | 2 tests | root::adds + lazy testset root::group |
| 0.18.1-nightly.20260909.a | 2 tests | root::adds + lazy testset root::group |
| 0.19.0 | 2 tests | [] |
0.19.0 with -F beta |
2 tests | [] |
| 0.19.1-nightly.20260915.a | 2 tests | [] |
So the regression landed between 0.18.1-nightly.20260909.a (released 2026-09-10) and 0.19.0 (released 2026-09-11).
We first saw it on a real project with 34 .baml files: the CLI lists 142 tests, the playground none. It happens both in the editor extension installed from the 0.19.0 toolchain's assets/baml-vscode.vsix and in the standalone baml playground, so it looks server side rather than UI side.
Reproduction Steps
Save both files below in one directory and run it with an installed toolchain version. The script creates a throwaway project with one top-level test and one testset, lists tests with the CLI, starts baml playground --no-open, and asks the server for the test tree over /api/ws, the same message the Tests panel sends. It needs Node 22 or newer for the global WebSocket.
./repro.sh 0.18.1-nightly.20260909.a # returns the tree
./repro.sh 0.19.0 # returns []Output on 0.18.1-nightly.20260909.a:
--- baml test --list
Selected 2 test(s)
root::adds (<testset>)
root::group::inner (<testset>)
--- baml playground
toolchain 0.18.1-nightly.20260909.a protocol 2
build generation 1 test init functions 1
testCollectionResult [{"$type":"testing.SerializedTest","type":"test","name":"root::adds"},{"$type":"testing.SerializedTest","type":"lazyTestSet","name":"root::group"}]Output on 0.19.0, and identically on 0.19.1-nightly.20260915.a:
--- baml test --list
Selected 2 test(s)
root::adds (<testset>)
root::group::inner (<testset>)
--- baml playground
toolchain 0.19.0 protocol 2
build generation 1 test init functions 1
testCollectionResult []To see it in the UI instead, open the printed project in baml playground on 0.19.0 and expand Tests. It shows "No tests found". "Re-collect tests" sends the same requestCollectTests message and gets the same empty tree.
repro.sh
#!/usr/bin/env bash
# Usage: ./repro.sh <toolchain-version> [port]
# Creates a two-test project pinned to <toolchain-version>, lists tests with the CLI,
# then starts `baml playground` and asks it for the test tree over /api/ws.
set -euo pipefail
VERSION=$1
PORT=${2:-4291}
HERE=$(cd "$(dirname "$0")" && pwd)
DIR=$(mktemp -d)
trap 'kill "${PG_PID:-}" 2>/dev/null || true; rm -rf "$DIR"' EXIT
mkdir -p "$DIR/baml_src"
printf '[package]\nname = "collect-repro"\n\n[toolchain]\nversion = "%s"\n' "$VERSION" > "$DIR/baml.toml"
cat > "$DIR/baml_src/main.baml" <<'BAML'
function add(a: int, b: int) -> int {
a + b
}
test "adds" {
assert.equal(add(1, 2), 3)
}
testset "group" {
test "inner" {
assert.equal(add(2, 2), 4)
}
}
BAML
cd "$DIR"
echo "--- baml test --list"
baml test --list
echo "--- baml playground"
baml playground --port "$PORT" --no-open > "$DIR/playground.log" 2>&1 &
PG_PID=$!
for _ in $(seq 1 60); do
if (exec 3<>"/dev/tcp/127.0.0.1/$PORT") 2>/dev/null; then break; fi
sleep 0.5
done
node "$HERE/collect-probe.mjs" "$PORT"collect-probe.mjs
// Usage: node collect-probe.mjs <port> (Node >= 22 for the global WebSocket)
// Connects to a running `baml playground`, asks for the test tree of every project
// once the build is current, and prints what the server sends back.
const port = process.argv[2] ?? '4265';
const ws = new WebSocket(`ws://127.0.0.1:${port}/api/ws`);
let projects = [];
let asked = false;
ws.onopen = () => ws.send(JSON.stringify({ type: 'requestState' }));
ws.onmessage = (ev) => {
const m = JSON.parse(ev.data);
if (m.type === 'hello') console.log('toolchain', m.toolchainVersion, 'protocol', m.playgroundProtocol);
const n = m.notification;
if (!n) return;
if (n.type === 'listProjects') projects = n.projects.map((p) => p.path);
if (n.type === 'updateProject' && n.update.isBexCurrent && !asked) {
asked = true;
const init = n.update.functions.filter((f) => f.name.startsWith('$init_test')).map((f) => f.name);
console.log('build generation', n.update.generation, 'test init functions', init.length);
for (const p of projects) ws.send(JSON.stringify({ type: 'requestCollectTests', project: p }));
}
if (n.type === 'testCollectionResult') {
const tree = new TextDecoder().decode(new Uint8Array(n.data ?? []));
console.log('testCollectionResult', tree, n.expandError ? `expandError=${JSON.stringify(n.expandError)}` : '');
ws.close();
process.exit(0);
}
};
setTimeout(() => { console.log('timeout: no testCollectionResult'); process.exit(2); }, 60000);Workaround: serve the playground with an older toolchain binary, for example ~/.baml/toolchains/0.18.1-nightly.20260909.a/bin/baml-cli playground, as long as the project still compiles there.
BAML Version
0.19.0 (also 0.19.1-nightly.20260915.a); baml wrapper 0.2.0
Language/Framework
Node.js
LLM Provider
Other
LLM Model
Not involved; the tests are pure BAML with no LLM call
Operating System
Linux
Browser
Chrome
Code Editor
Other (Antigravity 1.107, a VS Code fork, with the extension from the 0.19.0 toolchain VSIX), and the standalone baml playground
Source: BoundaryML/baml