Generate the skill catalog instead of hand-maintaining it across surfaces
Summary
The skill list is maintained by hand across several tracked files, and nothing in CI checks that those copies agree with skills/. Adding a skill means remembering every surface. There is currently drift on main because one of those surfaces was missed.
Proposal: derive the catalog surfaces from skills/ with a script that has a --check mode, and run that check in CI. Every existing script under scripts/ validates; none generates. This is the first case where generating is cheaper than remembering.
Current drift on main
constraint-driven-development shipped in #472 (2ce8d47, 28 Aug) and is missing from two places:
| Surface | State |
|---|---|
README.md |
present |
skills/using-agent-skills/SKILL.md |
present |
CLAUDE.md, "Skills by Phase" (line 23) |
missing |
.github/ISSUE_TEMPLATE/skill-gap.yml dropdown |
missing |
The issue-form gap has a user-visible consequence: someone hitting bad guidance from constraint-driven-development today cannot select it when filing a skill-gap report. The form landed in b68f553 (19 Jul), the skill five weeks later, and nothing connected them.
Why this is a process gap and not one contributor's slip
#472 was not careless. It updated ten files across the catalog:
README.md
skills/using-agent-skills/SKILL.md
docs/adoption-guide.md
docs/codex-setup.md
docs/commandcode-setup.md
docs/comparison.md
evals/cases/constraint-driven-development.json
commands/constraints.toml
.claude/commands/constraints.md
.gemini/commands/constraints.toml
It still missed two. Ten hand-updated surfaces per skill is past the point where care is the right mechanism, and the number grows with every runtime integration the repo adds (#530, #355, #439, #477 are all open requests that would add more).
Sketch
scripts/sync-catalog.js, reading skills/*/SKILL.md frontmatter as the single source of truth:
--check(CI): exit non-zero listing any surface whose skill set disagrees withskills/--write(local): update the generated regions in place
Generated regions marked with comment fences so hand-written prose around them is untouched, for example <!-- catalog:start --> / <!-- catalog:end --> in Markdown and the equivalent in the issue form's options: block.
Starting with --check alone would already have caught this, and is the smaller change if writing into tracked docs feels too invasive for a first cut.
Open questions
- Which surfaces should be authoritative-generated versus merely checked?
README.mdprose probably wants a human touch; the issue-form dropdown is pure data and could be fully generated. - Does
CLAUDE.md's phase grouping belong in frontmatter (aphase:field) or in a small mapping file the script reads? The phase is currently implicit in prose. - Should this live in an existing validator rather than a new script?
validate-skills.jsalready walksskills/, so the check could be a mode there instead.
Not overlapping with existing work
#137 covers markdown formatting consistency (markdownlint, heading levels, list markers), which is orthogonal to catalog membership. #511 is about doc/code drift as guidance inside the documentation-and-adrs skill, not this repo's own catalog. I did not find an open PR touching this.
Credit
The generate-plus---check pattern is borrowed from affaan-m/ecc, which ships catalog:check / catalog:sync and command-registry:generate --check for the same problem at a larger catalog size. The idea is theirs; no code is proposed to be copied.
Happy to open a PR for the --check half if the direction is right. Given that scripts/ is collision-prone with in-flight work (#428, #444, #517 all touch validators), I would rather agree the shape here first.
Source: addyosmani/agent-skills