fix: AnthropicLLM uses deprecated completions API, broken for Claude 3+ and drops full conversation history
Author: immuhammadfurqanCreated Jun 24, 2026Updated Jun 29, 2026
Bug Description
application/llm/anthropic.py uses the legacy Claude 2-era text completions API:
# lines 47-52
completion = self.anthropic.completions.create(
model=model,
max_tokens_to_sample=max_tokens,
prompt=f"{self.HUMAN_PROMPT} {prompt}{self.AI_PROMPT}",
)This API is not supported by any Claude 3/3.5/4 model. Additionally both _raw_gen and _raw_gen_stream only extract messages[0] and messages[-1], silently discarding the entire multi-turn conversation history between them. All other LLM providers pass the full messages list.
Expected Behavior
AnthropicLLM should use client.messages.create() / client.messages.stream() (the modern Messages API), pass the full conversation history, and correctly handle system messages via the system= parameter.
Proposed Fix
- Remove deprecated
HUMAN_PROMPT/AI_PROMPTimports and instance variables - Rewrite
_raw_gento useanthropic.messages.create(), returnresponse.content[0].text - Rewrite
_raw_gen_streamto useanthropic.messages.stream(), yield text deltas - Add
_split_messages()helper to separate system prompt from conversation turns
Source: arc53/DocsGPT