[Gartenberg sub A] Deterministic writing-surface measurement (readability, nominalization, jargon, hedging) as an advisory on drafts and reviewer outputs
Author: Imbad0202Created Sep 7, 2026Updated Sep 8, 2026
Labelsenhancementpaper-derived
Sub-issue of [META] Gartenberg et al. 2026 (#829)
Anchor
- Editorial §3.3 and Fig. 8 (manuscripts), §4.1 and Fig. 11 (reviews): AI share is associated with lower Flesch Reading Ease, higher grade level, FOG and SMOG, more nominalizations ("conceptualization", "operationalization"), more jargon; and with less passive voice, less hedging, and more specificity. Tables 2-3: readability predicts desk rejection separately from AI share; the all-non-native-English × AI interaction is ≈ 0, so AI writing does not help the authors most likely to rely on it.
- ARS files:
academic-paper/references/writing_quality_check.md(flagged terms, em-dash and semicolon caps, throat-clearing openers, burstiness; rules only, nothing measured),academic-paper/agents/draft_writer_agent.mdStep 2.7 self-review,academic-paper/agents/formatter_agent.md, the reviewer panel output surfaces,shared/references/claim_strength_ladder.md(#569).
Problem
ARS asks the writer to apply good-prose rules but never measures the result. The editorial shows the measurable surface that editors actually react to (readability, nominalization density, jargon) and shows it degrading in both manuscripts and reviews. Two ARS surfaces are exposed: the draft the user submits, and the reviewer panel's own reports, which the user has to read and act on (the editorial's §4.1 point: unreadable reviews raise author and editor effort). The hedging finding cuts the other way and is worth recording: AI text hedges less, which is the direction the #569 ladder already guards against as claim-strength drift.
Proposed change
- A stdlib-only script (
scripts/check_writing_surface.py, name open) that reports per section and per document: Flesch Reading Ease, Flesch-Kincaid grade, FOG, SMOG, nominalization share (suffix list, ARS-owned), passive-voice share (heuristic, labelled as such), hedge-term density (reuse the ladder's hedge vocabulary), and sentence-length dispersion (the burstiness rule already in the checklist, now measured). English only in the first cut; zh-TW needs different measures and is out of scope here. - Advisory rows (
ADV-WS-<n>) at the draft self-review step and at the formatter's final pass, and the same report on each reviewer seat's output and the decision letter. Advisory only: no threshold gate, no refusal. - Baseline before any threshold: run the script over the existing held-out outputs (
evals/heldout/revision_claim_drift/,evals/heldout/reviewer_seeded_defects/) and record the distribution, so a later threshold is measured rather than borrowed from the editorial's journal.
Acceptance
- Script with tests (fixture texts with known index values; the nominalization and hedge lists are repo-owned files, not hardcoded)
- Advisory row emission wired at the three surfaces above, with the canonical enforcement sentence (advisory, never gating)
- Baseline distribution recorded under
evals/with the frozen commit; no threshold shipped in this issue -
writing_quality_check.mdgains a short "what is measured vs. what is a rule" section and keeps its "not a humanizer, not detection evasion" boundary verbatim
Non-goals
- No AI-likelihood score of any kind. Readability indices are craft measures, not detection.
- No automatic rewriting; the writer sees the rows and decides.
- No claim that lower indices mean better science; the editorial's own caveat (readability is one dimension of quality) carries over.
Source: Imbad0202/academic-research-skills