browseros-mcp: UTF-8 snapshot truncation can infinite-loop and pin a CPU core

Summary

browseros-mcp::token_estimate::slice_text_by_estimated_tokens() can enter an infinite loop when the binary-search midpoint falls inside a multibyte UTF-8 character.

For large non-ASCII snapshots/diffs this can pin one Tokio worker at ~100% CPU indefinitely. Two concurrent affected dispatches can therefore hold browseros-claw-server around ~200% CPU, and session cancellation cannot drain them because the loop has no async/cancellation point.

I reproduced this against claw-server/v0.0.50, and the same implementation is still present at current BrowserOS source commit 96ff75aa8f3f023c526308df32cdd299331a3ec9.

Root cause

Current code:

let mut low = 0;
let mut high = text.len();
while low < high {
    let mid = (low + high).div_ceil(2);
    let candidate = floor_char_boundary(text, mid);
    if estimate_text_tokens(&text[..candidate]) <= max_tokens {
        low = candidate;
    } else {
        high = candidate.saturating_sub(1);
    }
}

If mid lands inside a multibyte character, floor_char_boundary(text, mid) can return the existing low. If that candidate is within the token budget, low = candidate makes no progress and the loop repeats forever.

A minimal case is 5001 Chinese characters with max_tokens = 5000:

• UTF-8 byte length: 15003
• eventually low=15000, high=15003
• mid=15002
• floor_char_boundary(..., 15002) = 15000
• low stays 15000 forever

Regression test

#[test]
fn slicing_multibyte_text_terminates_on_a_utf8_boundary() {
    let text = "汉".repeat(5001);
    let sliced = slice_text_by_estimated_tokens(&text, 5000);
    assert_eq!(estimate_text_tokens(&sliced), 5000);
    assert_eq!(sliced, "汉".repeat(5000));
}

On the current implementation the call does not return.

Minimal fix

Because the estimator is currently exactly ceil(utf8_bytes / 3), the binary search is unnecessary. The maximum admissible byte length is simply max_tokens * APPROX_CHARS_PER_TOKEN; clamp that to the input length and floor once to a UTF-8 boundary:

pub fn slice_text_by_estimated_tokens(text: &str, max_tokens: usize) -> String {
    if estimate_text_tokens(text) <= max_tokens {
        return text.to_string();
    }

    let max_bytes = max_tokens
        .saturating_mul(APPROX_CHARS_PER_TOKEN)
        .min(text.len());
    let end = floor_char_boundary(text, max_bytes);
    text[..end].to_string()
}

With that change, the multibyte regression passes and the existing token_estimate test group remains green.

Impact observed

This is especially easy to hit on long CJK ChatGPT pages because snapshot/diff formatting calls this helper once output exceeds the inline token threshold. A stuck synchronous formatter also prevents the owning dispatch from reacting to cancellation, so POST /api/v1/sessions/{id}/cancel can remove the session from the live projection while the request itself waits indefinitely for the active dispatch to drain.