generate_plugins_json.py silently drops plugins when GitHub rate-limits it (and plugins.json is 8 days stale)

I went to send a follow-up PR regenerating dashboard/public/plugins.json and stopped, because the generator produced a file that looked fine and was missing five plugins. The regeneration is worth doing, but I do not think it should be done by hand until this is fixed.

1. generate_plugins_json.py drops entries silently under GitHub's secondary rate limit

gh_api() retries three times on a rate-limit response — 30s, 60s, 90s — and then returns None:

if "rate limit" in result.stderr.lower():
    wait = 30 * (attempt + 1)
    print(f"  ⏳ Rate limited. Waiting {wait}s...")
    time.sleep(wait)
    continue
...
return None

Callers treat None as "this repo has no data", so the plugin is dropped from the output and the script still exits 0 and writes a syntactically valid file. Nothing downstream can tell the difference between "this plugin is gone" and "GitHub said slow down".

I ran the generator three times today. Numbers from the actual runs:

Run 2's five dropped entries:

thedotmack/claude-mem
jarrodwatts/claude-hud
EveryInc/compound-engineering-plugin
alirezarezvani/claude-skills
davepoon/buildwithclaude

claude-mem alone is ~92k stars. Committing run 2 would have removed all five from the marketplace, and the diff would have looked like a routine regeneration.

It is the secondary rate limit, not the hourly quota. gh api rate_limit reported 5000/5000 remaining throughout. gh_api() shells out to the gh CLI once per call and only sleeps every 50 requests, which is fast enough to trip GitHub's abuse protection regardless of quota. Adding a 0.35s delay between calls was not enough — run 3 hit the limit on the first repository.

Suggested fix

Make the exhausted-retry path fail loudly instead of returning None:

raise RuntimeError(f"gh api {endpoint} failed after {retries} attempts")

or, if a partial run should still be allowed, track failures and refuse to write the output when any occurred — the script already counts them for its summary line, so the information is there; it just is not acted on.

A per-request delay would reduce how often this happens, at a cost: run 1 made 3,333 requests and took about 19 minutes as it is.

2. Separately: #845 is merged but the entry is not live

iOSDevSK/html2wp-cc-plugin was added to PLUGIN_SOURCES in #845 and merged on 29 Aug. It does not appear on the plugins page, because dashboard/public/plugins.json is a committed artifact read at build time:

// dashboard/src/pages/plugins/index.astro
const filePath = nodePath.join(process.cwd(), 'public', 'plugins.json');

and nothing regenerates it — update-json-data.yml runs generate_components_json.py, generate_trending_data.py and generate_claude_jobs.py on its daily cron, but not generate_plugins_json.py. The committed file was last written on 22 Aug in #827 and lists 34 plugins.

#827 happened to update both the script and the JSON in one commit, so it worked. A PR that only adds the source line — the natural reading of PLUGIN_SOURCES — silently does nothing visible. Worth a line in the contributing docs either way.

Happy to help

I can send the fix in (1) as a PR, and regenerate plugins.json once it fails loudly enough to trust. Or if you would rather run the generator yourself with a token that has more headroom, that solves (2) immediately and I will just send the (1) patch.