Local API server (port 1337) drops connection without response on certain Cyrillic request bodies (non-char-boundary slice panic)

Local API server (port 1337) drops the connection on certain Cyrillic request bodies — likely a non-char-boundary string slice panic

Summary

The Jan local API server on 127.0.0.1:1337 closes the connection without sending any response for certain /v1/chat/completions requests whose body contains multi-byte UTF-8 text (e.g. Cyrillic). The client sees Server disconnected without sending a response (httpx RemoteProtocolError, or ECONNRESET / empty reply from curl).

It is not "any Cyrillic" and not an encoding problem on the client. Whether a request crashes depends on the exact byte length/alignment of the request body: shifting the body by a single ASCII byte flips the request between "works" and "crashes" deterministically. This is the classic signature of slicing a Rust String/&str at a byte index that is not on a UTF-8 char boundary (byte index N is not a char boundary), panicking the request handler.

The crash is in the Jan API server / proxy layer, before the request is forwarded to the model backend — see logs below. The underlying llama-server handles the very same prompt fine when called directly.

Environment

• Jan version: 0.8.3
• OS: Arch Linux, kernel 6.19.9-zen1 (x86_64)
• GPU: NVIDIA GeForce RTX 4050 Laptop (CUDA 13 backend, llama.cpp build b9743 / b9244)
• llamacpp-router/server version: 25.443.292
• Model used in repro: Jan-v3_5-4B-Q4_K_M (also reproduces on Jan-v2-VL-high-Q4_K_M — model-independent)
• Reached via the OpenAI-compatible endpoint http://127.0.0.1:1337/v1/chat/completions.

Steps to reproduce

POST a chat completion to http://127.0.0.1:1337/v1/chat/completions with a Cyrillic system message. Minimal curl:

# CRASHES: connection reset, no response
curl -sS http://127.0.0.1:1337/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Jan-v3_5-4B-Q4_K_M","max_tokens":16,"messages":[
        {"role":"system","content":"Ты извлекаешь финансовую"},
        {"role":"user","content":"Hello"}]}'

# WORKS: add a single leading space to the system content (shifts byte alignment)
curl -sS http://127.0.0.1:1337/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Jan-v3_5-4B-Q4_K_M","max_tokens":16,"messages":[
        {"role":"system","content":" Ты извлекаешь финансовую"},
        {"role":"user","content":"Hello"}]}'

A Python reproducer covering several cases is attached (jan_bugreport_repro.py).

Observed matrix (deterministic, 5/5 each)

Byte-alignment sweep — prepend k ASCII spaces to "Ты извлекаешь финансовую":

The pass/fail pattern tracks the byte offset, not the words — strong evidence of a fixed-offset slice landing inside a 2-byte Cyrillic character.

Expected behavior

The request is processed normally (HTTP 200) regardless of where multi-byte UTF-8 characters fall in the body; or, at worst, a proper HTTP error — never a silent connection drop.

Logs

~/.local/share/Jan/data/logs/app.log for a crashing request shows only the entry line and then nothing — the handler dies before extracting the model / forwarding upstream:

[app_lib::core::server::proxy][INFO] Handling POST request to /chat/completions requiring model lookup in body

A succeeding request shows the full chain:

[app_lib::core::server::proxy][INFO] Handling POST request to /chat/completions requiring model lookup in body
[app_lib::core::server::proxy][DEBUG] Extracted model_id: Jan-v3_5-4B-Q4_K_M
[app_lib::core::server::proxy][DEBUG] Routing model_id Jan-v3_5-4B-Q4_K_M via llamacpp router
[app_lib::core::server::proxy][INFO] Proxying request to model server at base URL http://127.0.0.1:50381/v1/chat/completions
...
[app_lib::core::server::proxy][DEBUG] Streaming complete to client

So the panic happens in the app_lib::core::server::proxy layer, between Handling POST request ... requiring model lookup in body and Extracted model_id — i.e. while reading/inspecting the body to look up the model. Calling the underlying llama-server directly (router port, with the api key) returns HTTP 200 for the identical prompt, confirming the bug is in the Jan proxy, not the model backend.

Likely cause / where to look

A byte-indexed slice of the request body (or of the extracted model-id / a logging snippet) in the proxy's "model lookup in body" path. Anything like &body[..n], body[..n].to_string(), split_at(n), or a manual byte-range read will panic when n is not a char boundary. Use char-boundary-safe operations (char_indices, floor_char_boundary, chars().take(n), or parse the JSON without slicing raw bytes).

Workarounds (for affected users)

• Keep the system prompt ASCII-only (put non-ASCII data in user messages) — avoids the fatal offset in practice.
• Or bypass the 1337 proxy and call the llama-server port directly with the api key (fragile: the port changes per launch).

Python reproducer (jan_bugreport_repro.py)

#!/usr/bin/env python3
"""
Minimal reproducer: Jan API server (port 1337) drops the connection without a
response on certain Cyrillic (multi-byte UTF-8) request bodies.

Run with Jan's local API server enabled on 127.0.0.1:1337.
"""
import httpx

URL = "http://127.0.0.1:1337/v1/chat/completions"
MODEL = "Jan-v3_5-4B-Q4_K_M"  # any loaded model reproduces it


def call(system, user):
    payload = {
        "model": MODEL,
        "messages": [
            {"role": "system", "content": system},
            {"role": "user", "content": user},
        ],
        "stream": False,
        "max_tokens": 16,
    }
    try:
        with httpx.Client(timeout=60) as c:
            r = c.post(URL, json=payload)
        return f"HTTP {r.status_code}"
    except Exception as e:
        return f"{type(e).__name__}: {e}"


cases = [
    ("ascii system",                "You extract info.",            "Hello"),
    ("cyrillic in USER only",       "You extract info.",            "Сапоги стоят 5 крон."),
    ("short cyrillic system OK",    "Ты помощник.",                 "Hello"),
    ("cyrillic system -> CRASH",    "Ты извлекаешь финансовую",     "Hello"),
    # +1 ASCII byte shifts alignment and it works again:
    ("same + 1 space -> OK again", " Ты извлекаешь финансовую",     "Hello"),
]

for name, sysmsg, usermsg in cases:
    print(f"{name:30s} -> {call(sysmsg, usermsg)}")