Generate the skill catalog instead of hand-maintaining it across surfaces

Author: nucliwebCreated Aug 31, 2026Updated Aug 31, 2026

Summary

The skill list is maintained by hand across several tracked files, and nothing in CI checks that those copies agree with skills/. Adding a skill means remembering every surface. There is currently drift on main because one of those surfaces was missed.

Proposal: derive the catalog surfaces from skills/ with a script that has a --check mode, and run that check in CI. Every existing script under scripts/ validates; none generates. This is the first case where generating is cheaper than remembering.

Current drift on main

constraint-driven-development shipped in #472 (2ce8d47, 28 Aug) and is missing from two places:

Surface State
README.md present
skills/using-agent-skills/SKILL.md present
CLAUDE.md, "Skills by Phase" (line 23) missing
.github/ISSUE_TEMPLATE/skill-gap.yml dropdown missing

The issue-form gap has a user-visible consequence: someone hitting bad guidance from constraint-driven-development today cannot select it when filing a skill-gap report. The form landed in b68f553 (19 Jul), the skill five weeks later, and nothing connected them.

Why this is a process gap and not one contributor's slip

#472 was not careless. It updated ten files across the catalog:

README.md
skills/using-agent-skills/SKILL.md
docs/adoption-guide.md
docs/codex-setup.md
docs/commandcode-setup.md
docs/comparison.md
evals/cases/constraint-driven-development.json
commands/constraints.toml
.claude/commands/constraints.md
.gemini/commands/constraints.toml

It still missed two. Ten hand-updated surfaces per skill is past the point where care is the right mechanism, and the number grows with every runtime integration the repo adds (#530, #355, #439, #477 are all open requests that would add more).

Sketch

scripts/sync-catalog.js, reading skills/*/SKILL.md frontmatter as the single source of truth:

  • --check (CI): exit non-zero listing any surface whose skill set disagrees with skills/
  • --write (local): update the generated regions in place

Generated regions marked with comment fences so hand-written prose around them is untouched, for example <!-- catalog:start --> / <!-- catalog:end --> in Markdown and the equivalent in the issue form's options: block.

Starting with --check alone would already have caught this, and is the smaller change if writing into tracked docs feels too invasive for a first cut.

Open questions

  1. Which surfaces should be authoritative-generated versus merely checked? README.md prose probably wants a human touch; the issue-form dropdown is pure data and could be fully generated.
  2. Does CLAUDE.md's phase grouping belong in frontmatter (a phase: field) or in a small mapping file the script reads? The phase is currently implicit in prose.
  3. Should this live in an existing validator rather than a new script? validate-skills.js already walks skills/, so the check could be a mode there instead.

Not overlapping with existing work

#137 covers markdown formatting consistency (markdownlint, heading levels, list markers), which is orthogonal to catalog membership. #511 is about doc/code drift as guidance inside the documentation-and-adrs skill, not this repo's own catalog. I did not find an open PR touching this.

Credit

The generate-plus---check pattern is borrowed from affaan-m/ecc, which ships catalog:check / catalog:sync and command-registry:generate --check for the same problem at a larger catalog size. The idea is theirs; no code is proposed to be copied.

Happy to open a PR for the --check half if the direction is right. Given that scripts/ is collision-prone with in-flight work (#428, #444, #517 all touch validators), I would rather agree the shape here first.