Coverage is computed but never affects the exit code: a partially-blind scan is indistinguishable from a complete one in CI

Summary

analysis_completeness already carries everything needed to know how much of a skill was inspected: coverage_percent, is_complete, ledger_exceptions with reason codes. It is a good design and it is why the numbers below exist at all.

But it stops at the JSON boundary. skillspector scan exits 1 only when risk_score > RISK_THRESHOLD, and 2 on error. A scan that inspected 86 % of the components exits 0 exactly like one that inspected 100 %. In CI, where the exit code is the whole interface, the two are indistinguishable.

Measured

91 reports, Claude Code skill marketplaces on one workstation:

The size_limit bucket is the one that changes conclusions: 73 distinct .js files above MAX_FILE_CHARS were never opened by any static analyzer. They are executable files inside installed skills. The ledger records this correctly — nothing is hidden — but no automated gate can act on it.

Padding a payload past 1 000 000 characters is not a sophisticated evasion.

Suggestion

Two opt-in flags, so nothing changes for existing users:

--min-coverage FLOAT     non-zero exit if coverage_percent < FLOAT
--fail-on-incomplete     non-zero exit if is_complete is false

A distinct exit code (say 3) would keep "risk found" and "could not look" separable — they lead to different actions.

One caveat from having built this in a wrapper: an absolute threshold fires on 88 reports out of 91 on day one, and then gets switched off. Accepting the current coverage into a baseline and failing on regression proved far more durable. That may be more than SkillSpector wants to own; a plain --min-coverage already unblocks the CI case.

Happy to send a PR.

Environment

SkillSpector 2.5.1 and main @ 2.9.5, Linux, Python 3.13.