Add a pinned LintLang gate for canonical authored SKILL.md files
What
Add a deterministic instruction-language lint as a non-fatal-by-default CI gate for the SKILL.md files this repository authors and owns directly.
Why
This repo's existing checks (structural link-sync, TypeScript typecheck, mock skill evaluations) verify plumbing and model behavior, but nothing checks the instruction text itself for known LLM-agent failure shapes — unbounded retry loops, missing constraint scaffolding, ambiguous tool descriptions, role confusion, etc. LintLang is a zero-LLM, deterministic static analyzer purpose-built for that (H1–H7 structural detectors, PASS/REVIEW/FAIL verdicts).
Exact scope
- Tool:
lintlang==0.6.0(pinned, latest release as of this proposal), invoked viauvx --from lintlang==0.6.0 lintlang scan <files> --fail-on fail. - Scanned files: only
SKILL.mdsources authored in this repository — the plugin trees this repo owns directly (azure-sdk-{dotnet,java,python,rust,typescript},deep-wiki,microsoft-365-agents-toolkit) plus the standalone.github/skills/*entries, de-duplicated against their plugin-tree originals. - Explicitly excluded:
azure-skillsandazure-kusto-graph-skills, both synced in fromgithub.com/microsoft/GitHub-Copilot-for-Azureper.claude-plugin/plugin.jsonand called out in CODEOWNERS as "synced from upstream." A gate here can't be fixed here — findings would need to go to that upstream repo instead.microsoft-foundryis naturally excluded too: every file under itsskills/is a symlink intoazure-skills. --fail-on fail: exits non-zero only on a CRITICAL/HIGH verdict. REVIEW-level findings print in the job summary but do not fail the build.
Verification already done (pre-PR)
Ran the exact pinned command against the current 156 canonical files on main (SHA 903dc62b1e4c833235b54db918a9a51cb6d3cc8f): 0 FAIL, 147 REVIEW, 9 PASS, exit 0 — the gate would not have blocked anything as of this scan. Separately, a temporary local negative-control file outside that canonical corpus, containing an unbounded-retry phrase ("keep trying until it succeeds"), reproduced a CRITICAL finding and a non-zero exit. That file is not part of the proposed workflow; this only confirms that the gate blocks an actually failing input.
Acceptance criteria
- Scoped to authored
SKILL.mdfiles only; no scan of upstream-synced content - No duplicate scanning across the
.github/skills↔.github/pluginsmirror - REVIEW visible, non-blocking; CRITICAL/HIGH blocking
- No
continue-on-error; no baseline file (current blocking-finding count is zero) - Zero edits to skill prose, mirrors, generated indexes, or lockfiles
- Existing structural/smoke/eval tests untouched and still passing
A PR implementing exactly this (one step added to test-harness.yml, one path-enumeration helper added to the existing .github/scripts/sync_skill_links.py) is ready to open once this shape is agreed on.
Source: microsoft/skills