revision-coach can infer committee authority from conference reviews (1 of 7 skill-fired runs took the #668 branch on ICLR reviews)

Author: Imbad0202Created Sep 12, 2026Updated Sep 12, 2026

Observed

During #853 verification (plugin-evals/03-iclr-rebuttal-en, with-plugin arm), 1 of 7 runs opened with: "you explicitly identified a real committee (ICLR reviewers), so this is the verbatim-source + concern-tracker + placeholder-skeleton path". The prompt never named a committee. The other 6 runs reasoned explicitly that anonymous conference referees are peer review and took the ordinary branch.

Both kinds of run also reported they could not read MODE_REGISTRY.md / references/* from the eval sandbox cwd, so the misrouting run improvised the committee artifacts from memory. Whether the same happens in a normal plugin install (where the Skill tool exposes the skill directory) is not verified.

Where the guard lives today

  • academic-paper/SKILL.md lines 331–334: "use the revision-coach variant only when the user explicitly identifies a real committee/institutional review office … do not infer official authority from tone".
  • academic-paper/agents/revision_coach_agent.md line 40: "Do not infer committee authority from tone or vocabulary."
  • SKILL.md line 304 lists "conference rebuttal" / "grant panel response" in the same trigger cell as "explicitly identified real committee correspondence", which may invite the conflation.

Suggested fix (for discussion)

Add one negative example to both guards: journal/conference peer reviewers, area chairs and programme committees are peer review, not a committee for this variant, unless the user says the letter comes from an institutional committee or review office. Consider splitting the line-304 cell so committee correspondence is its own row.

Acceptance

Add a display-only regex to plugin-evals/03 that fails on "committee-correspondence" / "#668" being applied (not merely mentioned as rejected); 0 of N runs take the branch.

Related

#851 / #853 (trigger fix that made this observable); #668 (committee variant).

Source: Imbad0202/academic-research-skills