#6319·openhuman

Sync History shows 0 tokens and $0.0000 because memory inference spend is never metered

Author: YellowSnnowmannCreated Sep 16, 2026Updated Sep 16, 2026
Labelsreact-uirust-corememorybugpriority: p1

Summary

Brain › Sync › Sync History renders every run with 0 tokens and $0.0000, and the header totals read 0 in / 0 out · $0.0000 total. The reader and the UI are correct — no writer anywhere puts a non-zero token or cost figure on a source-sync audit row. Underneath it is a wider gap: memory inference spend never reaches the host cost tracker at all, so it is also missing from openhuman.cost_get_dashboard and from budget accounting.

Problem

What happened — 12 sync runs listed, Items correct (50, 50, 0, 50…), Tokens 0 and Cost $0.0000 on every row.

What was expected — real provider-reported token counts and cost for runs that embedded 506 chunks against cloud voyage-3-large.

Impact

  • The panel states a sync was free. That is the one answer indistinguishable from the truth, and it is never true for a cloud embedder.
  • AgentActivityPanel's monthly memory cost (openhuman.memory_sources_monthly_cost_summary) is $0.00 for the same reason.
  • Memory spend is absent from the cost dashboard and from CostTracker::check_budget, which agent/tinyagents/host/budget_gate.rs uses for budget refusal and accounting. Memory inference is currently unbudgeted and unreported.

Steps to reproduce

  1. Connect a Composio source (GitHub/Gmail/Notion/Slack) with a cloud embedder configured.
  2. Brain › Sources › Sync on that source; wait for the run to finish.
  3. Brain › Sync › Sync History → the new row shows Items: 50, Tokens: 0, Cost: $0.0000.
  4. Settings › Agent Activity → monthly memory cost $0.00.

Environment — dev build on main @ 2475ea342, tinymemory pin v1.16.0, macOS. Staging profile ~/.openhuman-staging/users/69dc8c37935c9a8c539b43f2.

Root cause

Four independent layers. All four must be addressed or the symptom survives.

L1 — host writer hardcodes zeros. crates/openhuman-core/src/memory/sources/run_history.rs:96-100HostRun::into_entry writes batches: 0, input_tokens: 0, output_tokens: 0, estimated_cost_usd: 0.0, and HostRun has no token fields to carry. Covers every Sync click, Apply-all and Composio run (10 of the 12 rows).

L2 — driver writer hardcodes the same. tinymemory crates/tinymemory-core/src/sources/sync.rs:182-184 and :228-230, the periodic folder/GitHub/RSS/web loop (the other 2 rows). Zero since the file was created at tinymemory v1.5.0.

L3 — usage is produced, then dropped mid-chain.

stage state
provider returns UsageInfo (tinymemory-api/src/host/usage.rs) real tokens + charged_amount_usd
tree::summarise::summarise (tinymemory-core/src/tree/summarise.rs:169-188) populates SummaryOutput.input_tokens
HostSummariser::summarise (tinymemory-core/src/engine/summariser.rs:52) returns .output, drops usage
bucket_seal.rs:274 / document_seal.rs:196 call .summarise(), not summarise_with_usage()
audit row zeros

summarise_with_usage() has exactly one production caller: tinycortex/memory/sync/rebuild.rs:208, the engine rebuild path. RealCostAccumulator exists and is tested — wired only to rebuild, never to source sync. Embeddings have no token accounting anywhere.

L4 — the host cost tracker is never called from memory. cost::record_provider_usage and cost::record_embedding_usage exist, persist durably, and feed openhuman.cost_get_dashboard. Neither has a single caller from modules/memory_host.rs or from inference/embeddings/.

L4c — tinyinference never surfaces embedding usage. EmbeddingModel::embed returns Result<Vec<Vec<f32>>> (tinyinference/src/embeddings/types.rs:51) and no file under tinyinference/src/embeddings/ parses a usage object. Every real embedding request routes here via TinyAgentsEmbeddingProvider (inference/embeddings/provider_trait.rs:86), including the managed cloud path, which is only a decorator over it (cloud_adapter.rs:153). The serde_json::Value decode in inference/embeddings/factory.rs belongs to DimensionAgnosticOpenAiProbe, the Test-connection probe — not production embedding traffic. So provider-reported embedding tokens cannot be obtained host-side.

L4b — UI drops the fields that could already be non-zero. SyncAuditPanel.tsx:216,297 sums and renders estimated_cost_usd only. Rust has SyncAuditEntry::effective_cost_usd() (actual_charged_usd.unwrap_or(estimated) + composio_cost_usd) and summarise_month already uses it; the panel does not. The TS interface at app/src/utils/tauriCommands/memoryTree.ts:1077-1099 does not even declare composio_cost_usd, actual_charged_usd or composio_actions_called, though the wire carries all three.

Why it regressed

  • #3110 / PR #3150 (2026-06-01) built real-cost accounting, landing it on memory_sync/sources/{github,rebuild}.rs — the legacy paths.
  • 96c370314 (#4794, TinyCortex migration) gutted github.rs 861→few lines and rebuild.rs 836→few; the accounting moved into tinycortex, where only rebuild kept it.
  • 4ce2a8c7c (#5246, "retire legacy sync compatibility paths") deleted the remainder.
  • memory_sources/sync.rs — the Sources path that survived and is now the only path — always wrote zeros, even pre-migration. It never received #3110's work.

#3110 is closed as completed against a code path that no longer exists.

Field proof

~/.openhuman-staging/users/69dc8c37935c9a8c539b43f2/workspace/state/memory_sync_runs.jsonl — 10 rows, plus 2 in memory_tree/sync_audit.jsonl = the 12 on screen.

{"timestamp":"2026-09-15T13:13:01Z","source_kind":"composio","scope":"github:ca_leUcIgnTw0DJ",
 "items_fetched":50,"batches":0,"input_tokens":0,"output_tokens":0,"estimated_cost_usd":0.0,
 "composio_actions_called":0,"composio_cost_usd":0.0,"actual_charged_usd":null,
 "duration_ms":35511,"success":true}

Every row identical in the cost half.

Solution

Measured numbers only — no item-count estimates. Both inference seams already execute in the openhuman host process: the module installs BusEmbeddingHost / BusChatHost (tinymemory-module/src/lib.rs:136-143) which call back over tinybus into EmbeddingCallbacks::embed and ChatCallbacks::complete in crates/openhuman-core/src/modules/memory_host.rs. ModelResponse already carries usage. So the host can meter 100% of memory inference spend with no contract change.

The chat half is reachable host-side; the embedding half is not, and embeddings are the dominant memory spend (a sync that embeds 506 chunks may summarise with the no-LLM fallback_summary and spend nothing on chat at all). So the embedding half sets the PR count.

Repo nesting that fixes the order: openhuman → vendor/tinyagentsvendor/tinyinference, with openhuman's root Cargo.toml patching tinyinference to vendor/tinyagents/vendor/tinyinference/crates/tinyinference. A tinyinference change therefore reaches openhuman only through a tinyagents pointer bump.

PR 1 — tinyinference

Add EmbeddingModel::embed_with_usage returning vectors alongside an optional provider-reported usage, with a default implementation delegating to embed and reporting None, so no existing implementor breaks. Parse the usage object in the OpenAI-compatible, Voyage and Cohere models (all three return it); Ollama and Noop keep the default. Release.

PR 2 — tinyagents

Bump the vendor/tinyinference submodule pointer to that release. No code.

PR 3 — openhuman

  1. Bump vendor/tinyagents, taking the new tinyinference through the existing patch.
  2. inference/embeddings/provider_trait.rs — call embed_with_usage and, when usage is reported, record_embedding_usage(provider, model, tokens, dims, vectors). embed()'s public signature is unchanged; a provider that reports nothing records nothing.
  3. modules/memory_host.rsChatCallbacks::complete maps ModelResponse.usageUsageInfo and calls record_provider_usage. build_token_usage already skips all-zero payloads and sets CostSource::ProviderCharged when the backend echoes a charge.
  4. No new tag needed: ai.tinyhumans.tinymemory.EmbeddingHost is memory-only, and on the chat side role already discriminates ("summarization" is special-cased at memory_host.rs:129).
  5. UI — Sync History header shows measured memory spend for the window; per-row Tokens/Cost render with a tooltip rather than a fabricated $0.0000. TS type gains the three missing fields plus an effectiveCostUsd() mirroring the Rust accessor.

Outcome: real memory spend visible in Sync History and in the existing cost dashboard, and memory spend starts counting toward budget accounting. Per-row attribution still blank.

Separable now: steps 3–5 of PR 3 depend on nothing upstream and can ship first as an openhuman-only PR. That stops the $0.0000 claim immediately and meters summarisation, but on a profile using fallback_summary it will record nothing until the embedding half lands.

PR 4 — tinymemory (only if per-row attribution is wanted)

EmbeddingHost::embed and ChatHost::complete gain a trailing scope: Option<String>, appended at tail (MINOR — wire slots are append-only); bucket_seal.rs / document_seal.rs pass the source scope they already hold. No accounting in this PR — the host does the metering. Release.

PR 5 — openhuman

Re-pin rides inside this PR (precedent: #6269 pinned tinyconnectors v0.10.0 in the fix PR). TokenUsage gains a scope field (host-internal type, no contract); sync_audit_log_rpc joins the ledger by scope and fills the per-row columns.

Why per-row needs PR 4+5

The bus calls carry role, provider, model — no source id. Sealing is a queued async job (mem_tree_jobs), batched across sources and running after the run's row is written, so time-window attribution across concurrent syncs would be a guess rather than a measurement.

Acceptance criteria

  • Repro gone — after a sync with a cloud embedder, memory spend is non-zero and provider-reported wherever it is shown; no surface claims $0.0000 for a run that cost money.
  • No fabricated numbers — a provider that omits usage records nothing and logs at debug; no item-count estimate is presented as a measurement.
  • Memory embedding and summarisation spend appears in openhuman.cost_get_dashboard.
  • Memory spend reaches CostTracker, so check_budget accounts for it.
  • SyncAuditPanel renders actual_charged_usd + composio_cost_usd when a row carries them, and (not $0.0000) when a row genuinely has no cost recorded.
  • Rows written before this change still parse and render — every added field serde(default).
  • Regression safety — unit coverage on the two new metering call sites and on the UI's zero/absent distinction.
  • Diff coverage ≥ 80% — Vitest + cargo-llvm-cov.

Related

  • #3110 — built real-cost accounting, landed on paths later deleted; closed completed.
  • #3150 — the implementing PR.
  • #4794 (96c370314), #5246 (4ce2a8c7c) — the migrations that removed the accounting.
  • #6257 / #6264 — made Sync History record runs at all; this issue is the cost half that stayed zero.
  • #5820 — the tree_ingest_failures partial verdict on the same row shape.