#2564·DocsGPT

fix: AnthropicLLM uses deprecated completions API, broken for Claude 3+ and drops full conversation history

Author: immuhammadfurqanCreated Jun 24, 2026Updated Jun 29, 2026

Bug Description

application/llm/anthropic.py uses the legacy Claude 2-era text completions API:

python
# lines 47-52
completion = self.anthropic.completions.create(
    model=model,
    max_tokens_to_sample=max_tokens,
    prompt=f"{self.HUMAN_PROMPT} {prompt}{self.AI_PROMPT}",
)

This API is not supported by any Claude 3/3.5/4 model. Additionally both _raw_gen and _raw_gen_stream only extract messages[0] and messages[-1], silently discarding the entire multi-turn conversation history between them. All other LLM providers pass the full messages list.

Expected Behavior

AnthropicLLM should use client.messages.create() / client.messages.stream() (the modern Messages API), pass the full conversation history, and correctly handle system messages via the system= parameter.

Proposed Fix

  • Remove deprecated HUMAN_PROMPT/AI_PROMPT imports and instance variables
  • Rewrite _raw_gen to use anthropic.messages.create(), return response.content[0].text
  • Rewrite _raw_gen_stream to use anthropic.messages.stream(), yield text deltas
  • Add _split_messages() helper to separate system prompt from conversation turns