Bug report: Multi-turn conversation generation injects empty chunkContent, assistant never sees the source text
Bug report: Multi-turn conversation generation injects empty {{chunkContent}}, assistant never sees the source text
Project: https://github.com/ConardLi/easy-dataset
Version: 1.7.2 (observed in the packaged desktop app, .next/server/chunks/5246.js and .next/server/chunks/8197.js)
Summary
When generating multi-turn conversations (DatasetConversations), the assistant and follow-up question prompts are built with an empty {{chunkContent}} even when a valid chunk exists, because of an inverted variable-initialization bug in getAssistantReplyPrompt / getNextQuestionPrompt. The model therefore never sees the source text: it either refuses to answer ("材料中并未包含相关记载" / "no such record in the provided material") or hallucinates quotes that do not exist in the source.
Single-turn QA generation is NOT affected (it passes text: chunk.content directly).
Root cause
In app/api/.../multiTurnConversation prompt helpers (compiled chunks 5246.js / 8197.js, module 75246):
async function c(e, {scenario, roleA, roleB, chunkContent: s, ...}, m = null) {
let y = ""; // BUG: should be `let y = s;`
if (s.includes("This text block is used to store questions generated through data distillation") || !s) {
y = "没有可用的参考资料,请根据自己的知识直接生成回复"; // fallback string only set in the EMPTY branch
}
return await P$(..., { ..., chunkContent: y, ... }); // normal non-empty chunkContent becomes ""
}y is initialized to "" and only assigned a value inside the "no material" branch. So:
- chunk non-empty (normal case) →
ystays""→{{chunkContent}}is replaced with an empty string → model sees "## 参考资料:" with nothing after it. - chunk empty/distillation placeholder →
ygets the fallback sentence (this branch is actually correct).
The same bug exists in getNextQuestionPrompt (next-round question generation), so both the reply and the follow-up question are affected.
Impact
- Every multi-turn conversation generated via the UI or the batch task API lacks source context.
- Observed behavior in practice: assistant replies "not found in the provided material" for facts that ARE in the chunk (e.g. 李勣 presented 佩刀/玉带 to 张文瓘's colleagues — the exact passage is in the chunk), or hallucinates details (e.g. invented "扶掖乃行"/"失履" for 苏味道; the source says 味道徒步赴逮,席地菜食).
- Any downstream training/eval dataset built from multi-turn conversations is silently degraded.
Suggested fix
One line per function — initialize y with the chunk content:
let y = s; // getAssistantReplyPrompt
let y = t; // getNextQuestionPromptor equivalently:
let y = s || (s.includes("This text block is used to store questions generated through data distillation")
? "没有可用的参考资料,请根据自己的知识直接生成回复" : "");Note the two compiled chunks both inline module 75246, so the fix must be applied to both copies (the second one uses variable name i for chunkContent in getNextQuestionPrompt).
Reproduction
- Create any question bound to a normal chunk.
- Generate a multi-turn conversation (rounds ≥ 2) via POST
/api/projects/:projectId/dataset-conversations. - Inspect
rawMessages→ assistant replies claim the material contains no such record, or quote text that is not in the chunk. - Fix applied → assistant quotes the source verbatim (e.g. 佩刀、玉带 / 若子才,无施不可,焉用赠).
Environment
- Easy Dataset 1.7.2 desktop app (Windows)
- Model: qwen3.5-plus (OpenAI-compatible)
Source: ConardLi/easy-dataset