#4523·tabby

prompt_template is ignored with openai/completion and vllm/completion — server receives raw prefix instead of the formatted FIM prompt

Author: MetalKnightCreated Jul 21, 2026Updated Jul 21, 2026
Labelsbug-unconfirmed

Environment ● Tabby v0.32.0, self-hosted via Docker (official tabbyml/tabby:latest image) ● Inference backend: external OpenAI-compatible server (AMD Lemonade Server, NPU backend), exposing /v1/completions ● Model: Qwen2.5-Coder-1.5B-Instruct-NPU ● Client: VS Code extension TabbyML.vscode-tabby v1.28.0, tabby-agent v1.9.0-dev

Config (config.toml)

[model.completion.http]
kind = "openai/completion"  # also reproduced with "vllm/completion"
model_name = "Qwen2.5-Coder-1.5B-Instruct-NPU"
api_endpoint = "http://<host>:13305/v1"
api_key = "***"
prompt_template = "<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
 
[model.embedding.http]
kind = "openai/embedding"
model_name = "nomic-embed-text"
api_endpoint = "http://<host>:11434/v1"

Expected behavior The prompt sent to the completion backend should wrap segments.prefix / segments.suffix using the configured prompt_template, producing something like: <|fim_prefix|>...prefix content...<|fim_suffix|>...suffix content...<|fim_middle|>

Actual behavior A generic literal placeholder <|FIM|> is inserted at the cursor position instead of the configured template. Captured from a real completion request logged by the VS Code extension:

...alt->insertValue(0.0, 10.0);
 
 WOSS_EXCEPTION<|FIM|>
 
 // Set...

Confirmed independently from the receiving server's own logs: for a request with segments.prefix = "container.", the backend reported prompt_length=2 — i.e. only the bare prefix word was received, with no FIM wrapper tokens at all.

Reproduction steps

  1. Configure model.completion.http with kind = "openai/completion" (or "vllm/completion") and a prompt_template containing {prefix}/{suffix} placeholders.
  2. Point it at any backend that logs/echoes the received prompt (or use debug_options: { "return_prompt": true } on POST /v1/completions).
  3. Trigger a completion (either via the VS Code extension or directly via Swagger UI).
  4. Observe that the prompt received by the backend contains the raw prefix/suffix, with a generic <|FIM|> marker instead of the configured template. Additional notes ● Reproduced identically with both kind = "openai/completion" and kind = "vllm/completion" — same placeholder behavior in both cases. ● This appears related to a previously reported discussion (#3323) about prompt_template not working with openai/completion — this report additionally confirms the same issue persists with vllm/completion.