#1805·evals

MakeMeSay get_content fails on its documented dict response type

Author: sylvesterkaczmarekCreated Aug 21, 2026Updated Aug 21, 2026

Describe the bug

evals.elsuite.make_me_say.utils.get_content() is annotated to accept either a dict or a CompletionResult:

python
def get_content(response: Union[dict, CompletionResult]) -> str:

It handles CompletionResult through get_completions(), but the fallback assumes attribute-style OpenAI response objects:

python
return response.choices[0].message.content

A normal dictionary response such as:

python
{"choices": [{"message": {"content": "hello"}}]}

therefore raises AttributeError even though dict is explicitly part of the public input type.

Expected behavior

Dictionary responses should return response["choices"][0]["message"]["content"]; existing CompletionResult and attribute-style response behavior should remain unchanged.

Proposed fix

Add an explicit dictionary branch and focused regression coverage for dict and CompletionResult inputs.