[Feature] Usage metrics dashboard for token, ACU, and cost tracking
Author: afonsoftCreated Aug 4, 2026Updated Sep 17, 2026
Labelsenhancementfrontenduiagent-canvas
## Problem Statement
OpenHands (Agent Canvas) currently lacks **centralized visibility** into LLM usage, costs, and resource consumption. This creates several challenges for users managing AI workloads:
### Current Gaps
1. **No Token Tracking**: Cannot monitor token consumption per session, conversation, or provider
2. **Cost Blindness**: No visibility into API costs across different LLM providers
3. **ACU Monitoring**: Agent Compute Units (ACUs) usage not tracked or reported
4. **No Budget Controls**: Cannot set spending limits or alerts
5. **Limited Historical Data**: No trends or analytics over time
6. **Multi-Session Visibility**: Cannot compare usage across different conversations
7. **Provider Breakdown**: No per-provider cost and usage analysis
### User Impact
- ⚠️ **Unexpected Costs**: Users receive surprise API bills
- ⚠️ **No Optimization**: Cannot identify high-cost sessions or inefficient patterns
- ⚠️ **Limited Planning**: Cannot forecast costs or plan budgets
- ⚠️ **No Accountability**: In team settings, cannot track per-user usage
## Proposed Solution
Implement a comprehensive **Usage Metrics Dashboard** that provides real-time and historical visibility into:
### Key Metrics
#### 1. Token Usage
- **Per Session**: Input/output tokens per conversation
- **Per Model**: Token breakdown by LLM model
- **Per Provider**: Token usage across different providers
- **Total Monthly**: Aggregate token usage with trend analysis
#### 2. ACU Consumption
- **Session ACUs**: Agent Compute Units per conversation
- **Daily Breakdown**: ACU usage by day
- **Monthly Total**: Total ACUs consumed in billing period
- **Rate Tracking**: ACUs per hour/day trends
#### 3. Cost Analysis
- **Real-time Costs**: Current session cost estimation
- **Daily Spend**: Cost breakdown by day
- **Monthly Budget**: Progress toward monthly budget with alerts
- **Provider Comparison**: Cost efficiency across providers
#### 4. Session Analytics
- **Duration**: Session length and idle time
- **Message Count**: User messages vs agent responses
- **Success Rate**: Completed tasks vs failures
- **Model Distribution**: Which models used most frequently
### Dashboard Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Usage Metrics Dashboard │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Tokens │ │ ACUs │ │ Cost │ │
│ │ 145.2K │ │ 8.3 │ │ $12.45 │ │
│ │ +12% ↑ │ │ +5% ↑ │ │ +8% ↑ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Token Usage Trend (Last 30 Days) │ │
│ │ ▁▂▃▅▄▃▅▆▇▆▅▄▃▅▆▇█▇▆▅▄▃▅▆▇▆▅▄▃▅▆▇▆▅▄ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────┐ ┌────────────────────────┐ │
│ │ Top Sessions │ │ Provider Breakdown │ │
│ │ 1. Debug API (12K) │ │ • Anthropic: $8.20 │ │
│ │ 2. Code Review (8K) │ │ • OpenAI: $3.15 │ │
│ │ 3. Refactor (5K) │ │ • Google: $1.10 │ │
│ └────────────────────────┘ └────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
## Implementation Approach
### Phase 1: Backend Metrics Collection
**New File**: `src/api/metrics-service.ts`
```typescript
export interface UsageMetrics {
sessionId: string;
conversationId: string;
userId?: string;
timestamp: string;
// Token metrics
inputTokens: number;
outputTokens: number;
totalTokens: number;
// ACU metrics
acuConsumed: number;
acuRate: number; // ACUs per hour
// Cost metrics
estimatedCost: number; // USD
provider: string;
model: string;
// Context
messageCount: number;
sessionDuration: number; // milliseconds
status: "active" | "completed" | "failed";
}
export interface AggregatedMetrics {
period: "day" | "week" | "month";
startDate: string;
endDate: string;
totalTokens: number;
totalACUs: number;
totalCost: number;
byProvider: Record;
byModel: Record;
bySessions: SessionSummary[];
}
export interface ProviderMetrics {
provider: string;
tokens: number;
acus: number;
cost: number;
requestCount: number;
}
export interface ModelMetrics {
model: string;
provider: string;
tokens: number;
cost: number;
averageLatency: number;
}
export interface SessionSummary {
sessionId: string;
conversationTitle: string;
startTime: string;
endTime: string;
tokens: number;
acus: number;
cost: number;
messageCount: number;
}
// Metrics collection
export async function recordMetrics(metrics: UsageMetrics): Promise {
await fetch("/api/metrics", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(metrics),
});
}
// Metrics retrieval
export async function getAggregatedMetrics(
period: "day" | "week" | "month",
startDate?: string,
endDate?: string
): Promise {
const params = new URLSearchParams({
period,
...(startDate && { startDate }),
...(endDate && { endDate }),
});
const res = await fetch(`/api/metrics/aggregated?${params}`);
if (!res.ok) throw new Error("Failed to fetch metrics");
return res.json();
}
export async function getSessionMetrics(
sessionId: string
): Promise {
const res = await fetch(`/api/metrics/sessions/${sessionId}`);
if (!res.ok) throw new Error("Session not found");
return res.json();
}
```
### Phase 2: Real-time Metrics Tracking
**New File**: `src/hooks/use-usage-metrics.ts`
```typescript
import { useState, useEffect, useCallback } from "react";
import { recordMetrics, type UsageMetrics } from "#/api/metrics-service";
export interface UseUsageMetricsOptions {
sessionId: string;
conversationId: string;
provider: string;
model: string;
}
export function useUsageMetrics(options: UseUsageMetricsOptions) {
const [metrics, setMetrics] = useState({
sessionId: options.sessionId,
conversationId: options.conversationId,
timestamp: new Date().toISOString(),
inputTokens: 0,
outputTokens: 0,
totalTokens: 0,
acuConsumed: 0,
acuRate: 0,
estimatedCost: 0,
provider: options.provider,
model: options.model,
messageCount: 0,
sessionDuration: 0,
status: "active",
});
const trackMessage = useCallback(
(data: { inputTokens: number; outputTokens: number; cost: number }) => {
setMetrics((prev) => {
const updated = {
...prev,
inputTokens: prev.inputTokens + data.inputTokens,
outputTokens: prev.outputTokens + data.outputTokens,
totalTokens: prev.totalTokens + data.inputTokens + data.outputTokens,
estimatedCost: prev.estimatedCost + data.cost,
messageCount: prev.messageCount + 1,
timestamp: new Date().toISOString(),
};
// Calculate ACU based on token usage
// Formula: ACU = (inputTokens * 0.00001) + (outputTokens * 0.00002)
const acuDelta =
data.inputTokens * 0.00001 +
data.outputTokens * 0.00002;
updated.acuConsumed = prev.acuConsumed + acuDelta;
return updated;
});
},
[]
);
const updateSessionDuration = useCallback((duration: number) => {
setMetrics((prev) => {
const updated = { ...prev, sessionDuration: duration };
// Calculate ACU rate (ACUs per hour)
if (duration > 0) {
updated.acuRate = (prev.acuConsumed / duration) * 3600000;
}
return updated;
});
}, []);
const completeSession = useCallback(
(status: "completed" | "failed" = "completed") => {
setMetrics((prev) => ({ ...prev, status }));
recordMetrics({ ...metrics, status });
},
[metrics]
);
// Auto-save metrics periodically
useEffect(() => {
const interval = setInterval(() => {
if (metrics.status === "active") {
recordMetrics(metrics);
}
}, 30000); // Every 30 seconds
return () => clearInterval(interval);
}, [metrics]);
return {
metrics,
trackMessage,
updateSessionDuration,
completeSession,
};
}
```
### Phase 3: Dashboard UI Components
**New File**: `src/components/features/metrics/usage-dashboard.tsx`
```typescript
import React from "react";
import { useQuery } from "@tanstack/react-query";
import { getAggregatedMetrics } from "#/api/metrics-service";
import {
MetricsCard,
TokenTrendChart,
ProviderBreakdown,
TopSessionsList,
CostAnalysis,
} from "./metrics-components";
export function UsageDashboard() {
const [period, setPeriod] = React.useState<"day" | "week" | "month">("month");
const { data: metrics, isLoading } = useQuery({
queryKey: ["metrics", "aggregated", period],
queryFn: () => getAggregatedMetrics(period),
refetchInterval: 60000, // Refresh every minute
});
if (isLoading) return ;
if (!metrics) return ;
return (
{/* Header */}
Usage Metrics
{/* Summary Cards */} {/* Token Trend Chart */} {/* Provider & Session Breakdown */} {/* Cost Analysis */} {/* Export & Budget Controls */} ); } ``` **New File**: `src/components/features/metrics/metrics-components.tsx` ```typescript export function MetricsCard({ title, value, trend, icon }) { return ( {icon} {title} {trend && ( {trend} )} {value} ); } export function TokenTrendChart({ data, period }) { // Use recharts or similar library for visualization const chartData = data.map((session) => ({ date: new Date(session.startTime).toLocaleDateString(), tokens: session.tokens, })); return (Token Usage Trend
); } export function ProviderBreakdown({ providers }) { const total = providers.reduce((sum, p) => sum + p.cost, 0); return (Provider Breakdown
{providers.map((provider) => ( {provider.provider} ${provider.cost.toFixed(2)} {formatNumber(provider.tokens)} tokens ))} ); } export function TopSessionsList({ sessions }) { return (Top Sessions
{sessions.map((session, idx) => ( #{idx + 1} {session.conversationTitle || "Untitled"} {formatNumber(session.tokens)} tokens · {session.messageCount} messages ${session.cost.toFixed(2)} ))} ); } export function BudgetAlertConfig({ currentSpend }) { const [budget, setBudget] = useState(100); const [alertThreshold, setAlertThreshold] = useState(80); const usagePercentage = (currentSpend / budget) * 100; const isAlertActive = usagePercentage >= alertThreshold; return (Budget Alert
Monthly Budget ($) setBudget(Number(e.target.value))} className="w-full mt-1 px-3 py-2 border rounded" /> Alert Threshold (%) setAlertThreshold(Number(e.target.value))} className="w-full mt-1 px-3 py-2 border rounded" /> {/* Progress Bar */} Current Usage ${currentSpend.toFixed(2)} / ${budget} {isAlertActive && (⚠️ You've reached {usagePercentage.toFixed(0)}% of your monthly budget
)} ); } ``` ### Phase 4: OmniRoute Integration (Optional) If Issue #1 (OmniRoute provider) is implemented, leverage OmniRoute's built-in metrics: **File**: `src/api/omniroute-metrics.ts` ```typescript export async function getOmniRouteMetrics( baseUrl: string, apiKey: string ): Promise { const res = await fetch(`${baseUrl}/api/combos/metrics`, { headers: { Authorization: `Bearer ${apiKey}` }, }); if (!res.ok) throw new Error("Failed to fetch OmniRoute metrics"); const data = await res.json(); // Transform OmniRoute metrics to OpenHands format return { period: "month", startDate: new Date().toISOString(), endDate: new Date().toISOString(), totalTokens: data.totalTokens || 0, totalACUs: calculateACUsFromTokens(data.totalTokens || 0), totalCost: data.totalCost || 0, byProvider: transformProviderMetrics(data.byProvider), byModel: transformModelMetrics(data.byModel), bySessions: [], }; } ``` ## Database Schema ### Metrics Table ```sql CREATE TABLE usage_metrics ( id SERIAL PRIMARY KEY, session_id VARCHAR(255) NOT NULL, conversation_id VARCHAR(255), user_id VARCHAR(255), timestamp TIMESTAMP NOT NULL DEFAULT NOW(), -- Token metrics input_tokens INTEGER DEFAULT 0, output_tokens INTEGER DEFAULT 0, total_tokens INTEGER DEFAULT 0, -- ACU metrics acu_consumed DECIMAL(10, 6) DEFAULT 0, acu_rate DECIMAL(10, 6) DEFAULT 0, -- Cost metrics estimated_cost DECIMAL(10, 4) DEFAULT 0, provider VARCHAR(100), model VARCHAR(255), -- Context message_count INTEGER DEFAULT 0, session_duration BIGINT DEFAULT 0, status VARCHAR(50) DEFAULT 'active', -- Indexes INDEX idx_session_id (session_id), INDEX idx_conversation_id (conversation_id), INDEX idx_user_id (user_id), INDEX idx_timestamp (timestamp), INDEX idx_provider (provider), INDEX idx_model (model) ); -- Aggregated metrics view for faster queries CREATE MATERIALIZED VIEW daily_metrics_summary AS SELECT DATE(timestamp) as date, provider, model, SUM(total_tokens) as total_tokens, SUM(acu_consumed) as total_acus, SUM(estimated_cost) as total_cost, COUNT(DISTINCT session_id) as session_count, SUM(message_count) as total_messages FROM usage_metrics GROUP BY DATE(timestamp), provider, model; -- Refresh materialized view daily CREATE OR REPLACE FUNCTION refresh_daily_metrics() RETURNS void AS $$ BEGIN REFRESH MATERIALIZED VIEW daily_metrics_summary; END; $$ LANGUAGE plpgsql; ``` ## Pricing Models ### Token-to-Cost Conversion ```typescript const PRICING: Record = { "anthropic/claude-opus-4": { input: 0.000015, output: 0.000075 }, "anthropic/claude-sonnet-4": { input: 0.000003, output: 0.000015 }, "openai/gpt-4": { input: 0.00003, output: 0.00006 }, "openai/gpt-4-turbo": { input: 0.00001, output: 0.00003 }, "google/gemini-2.5-pro": { input: 0.00000125, output: 0.00000375 }, }; exSource: OpenHands/OpenHands