Bug: check_primary_sources regex crosses lines (DOTALL) and scans wrong text range — wrong primary-source ratio

Author: wind-whCreated Sep 8, 2026Updated Sep 8, 2026

Bug: check_primary_sources regex mismatches across lines (DOTALL), scanning wrong text range

Environment

  • nuwa-skill v1.0.6 (installed via BuiltinMarket on Windows)
  • scripts/quality_check.py, function check_primary_sources()
  • Python 3.13.12 / Windows 11

Symptom

The "一手来源占比" (primary source ratio) check reported a wrong count (2/12 = 17%) against a SKILL.md whose appendix actually contains 7 primary + 6 secondary source items. After fixing the detection, the real ratio is 52% (PASS). The check was reading the wrong text range.

Root cause

python
source_section = re.search(
    r'(?:##\s+.*来源|## Source|## Reference)(.*?)(?=\n##\s|\Z)',
    content, re.DOTALL | re.IGNORECASE
)

Two compounding problems:

  1. . with re.DOTALL matches newlines, so ##\s+.*来源 can start at the first ## heading in the whole document and swallow everything up to the word 来源 — far away from the intended appendix heading.
  2. The heading pattern ##\s+ is not anchored to line start, so it also matches inside ### 一手来源 / ### 二手来源 (matching the last ## of the ### prefix). Combined with the lazy group and the lookahead, the engine picks a match whose capture group starts right after the ### 二手来源 heading — i.e. source_text ends up containing only the secondary-sources list, systematically deflating the primary ratio.

Reproduction

Any SKILL.md where the string 来源 appears in an early heading (e.g. ## 附录:调研来源) and sources are split into ### 一手来源 / ### 二手来源 subsections. Concretely: with 7 primary items under ### 一手来源 and 6 secondary items under ### 二手来源, the function reports 2/12 (17%) FAIL instead of counting all 13 items.

Suggested fix

Anchor to line start with re.MULTILINE, and bound the section at the next same-level (##) heading so ### subsections stay included:

python
def check_primary_sources(content: str) -> tuple[bool, str]:
    """检查一手来源占比"""
    # 行首锚定定位来源 section,避免 DOTALL 跨行误匹配
    m = None
    for mm in re.finditer(r'^##\s+.*(?:来源|Source|Reference).*$', content, re.MULTILINE | re.IGNORECASE):
        m = mm
        break
    if not m:
        return True, "未找到来源section(跳过检查)"

    rest = content[m.end():]
    # section 范围 = 直到下一个同级(##)标题;### 子节(一手/二手来源)计入本 section
    nxt = re.search(r'^##\s', rest, re.MULTILINE)
    source_text = rest[:nxt.start()] if nxt else rest

    primary = len(re.findall(r'一手|primary|本人著作|原始', source_text, re.IGNORECASE))
    secondary = len(re.findall(r'二手|secondary|转述|评论', source_text, re.IGNORECASE))
    total = primary + secondary
    if total == 0:
        return True, "未标记来源类型(跳过检查)"

    ratio = primary / total
    passed = ratio > 0.5
    return passed, f"一手来源占比: {primary}/{total} ({ratio:.0%}) {'✅' if passed else '❌ (应>50%)'}"

Verification

  • Real-world SKILL.md (7 primary + 6 secondary): before 2/12 (17%) FAIL → after 15/29 (52%) PASS (keyword hits across both subsections).
  • Boundary case, low ratio (1 primary + 3 secondary items): correctly FAIL.
  • Boundary case, high ratio: correctly PASS.

Fix currently applied locally; happy to send a PR if useful.