[Bug]: Mastery accepts card-delivery claims without posting a question card
Do you need to file an issue?
- I have searched the existing issues and this bug is not already filed.
- I believe this is a legitimate bug, not just a question or feature request.
Describe the bug
In an active Mastery Path study session, the tutor repeatedly claims that it has posted or reopened a question card, but finishes without calling mastery_quiz. There is no pending question or question-card event. Reminding it that no card is visible can produce the same unsupported claim again.
The existing finish guard handles plain A-D quizzes and some dangling question announcements, but accepts complete delivery statements such as 卡片这次重开一遍,题面我直接写在题干里了。 ("I have reopened the card and put the question in it"). Thus the existing correction round is never requested for these replies.
Steps to reproduce
- Open an existing, correctly bound Mastery Path in study mode, with no pending question.
- Ask for a diagnostic question or remind the tutor that its promised card is absent.
- The model calls
mastery_status, then returns a delivery claim such as the example above withfinish_reason=stopand no tool call. - The turn is accepted as complete, but there is no question card to answer.
Model wording is nondeterministic. A deterministic backend reproduction is to pass that reply to
MasteryLoopCapability.finish_instructionin an active mastery context with no successful card callback: ondevat897fce52, it returnsNoneinstead of a correction instruction. This also occurs when a previous question was graded in the same turn.
Expected Behavior
A current-turn claim of question-card delivery should require a successful card-posting callback. Otherwise, use the existing bounded correction path to call mastery_quiz, reusing an existing pending question where appropriate. If correction fails, do not retain the unsupported claim as a successful final answer. Ordinary teaching, quotations, historical reports and honest failure statements should still finish normally.
Related Module
Guided Learning
Configuration Used
Mastery Path study mode; Chinese data-structures tutoring; model identifier deepseek-flash; correctly bound path and session. The local observations were not caused by output truncation, exhausted tool budget or a native-tool-schema fallback.
Logs and screenshots
Sanitized summary of five consecutive observed failures:
mastery_status: study mode, correct path, next action probe, pending_question=null
assistant: claims a question card is posted/reopened
finish_reason=stop, tool_call_count=0
no mastery_quiz result or mastery_question cardA subsequent successful turn in the same session called mastery_quiz and produced a persisted question/card, distinguishing these failures from a general card-rendering outage. All five full failed replies were replayed against the guard locally. Private session identifiers and full learning history are omitted.
Additional Information
- DeepTutor Version: observed on installed 1.6.8; deterministic guard reproduction on upstream
devat897fce52. - Operating System: Windows.
- Python Version: 3.11.15.
- Node.js Version / Browser: not required for the backend reproduction.
- Related Issues: #1411, #941.
- Related implementation: merged #1002 introduced the finish-guard mechanism;
ce3532bsupplies the guarded-finish correction handling. This case never reaches that handling because the wording is not recognized. - #1416 /
2ad1d45and #1403 address handoff/session binding. The session here is already correctly bound, so those changes do not cover this reproduction. - Related completion gap: after actually selecting
mastery_quiz, a failed tool call could also be followed by arbitrary successful final prose, including through the budget-exhaustion forced-finish path. PR #1474 adds a turn-local selected-quiz obligation checked against the successful card callback. This state-based check covers attempted delivery; the status-only reproduction above still needs the supplementary text heuristic because the current protocol has no authoritative per-turn quiz intent before tool selection.NextStep.actionis advisory, so it must not force every teaching turn to post a card.
Source: HKUDS/DeepTutor