Baike.dev
Log in
Back to tool/Back to issues
#695·skills

Repository Health Dashboard

Author: github-actions[bot]Created May 27, 2026Updated Sep 16, 2026
Labelsdevops-health

Daily Health Check — 2026-08-31

Status: 6 critical · 5 warnings · 0 info Since yesterday: 5 new · ✅ 4 resolved · 4 unchanged

New Findings (5)

These appeared since the last health check (2026-08-30).

Evaluation workflow failed on main: vally (dotnet-blazor--claude-opus-5) — Run vally evaluations

  • Fingerprint: pipeline:evaluation:vally-(dotnet-blazor--claude-opus-5):run-vally-evaluations:failure
  • Details: Run #8011 "schedule: newer" (schedule event on main, created 2026-08-30T07:09Z) failed in job evaluate / vally (dotnet-blazor--claude-opus-5) at the Run vally evaluations step.
  • Action: Inspect vally evaluation logs for this shard/model combination.
  • Investigation: dispatched (see table below).

Evaluation workflow failed on main: vally (dotnet-data--claude-opus-5) — Run vally evaluations

  • Fingerprint: pipeline:evaluation:vally-(dotnet-data--claude-opus-5):run-vally-evaluations:failure
  • Details: Same run #8011 — job evaluate / vally (dotnet-data--claude-opus-5) failed at the Run vally evaluations step.
  • Action: Inspect vally evaluation logs for this shard/model combination.
  • Investigation: dispatched (see table below).

Evaluation workflow failed on main: vally (dotnet-maui--claude-opus-5) — Run vally evaluations

  • Fingerprint: pipeline:evaluation:vally-(dotnet-maui--claude-opus-5):run-vally-evaluations:failure
  • Details: Same run #8011 — job evaluate / vally (dotnet-maui--claude-opus-5) failed at the Run vally evaluations step.
  • Action: Inspect vally evaluation logs for this shard/model combination. (Not dispatched — budget cap of 2 reached; prioritize alongside dotnet-blazor/dotnet-data above.)

Evaluation workflow failed on main: vally (dotnet-diag--claude-sonnet-5) — Run vally evaluations

  • Fingerprint: pipeline:evaluation:vally-(dotnet-diag--claude-sonnet-5):run-vally-evaluations:failure
  • Details: Same run #8011 — job evaluate / vally (dotnet-diag--claude-sonnet-5) failed at the Run vally evaluations step.
  • Action: Inspect vally evaluation logs for this shard/model combination. (Not dispatched — budget cap of 2 reached.)

Issue Triage workflow failed: Execute GitHub Copilot CLI step

  • Fingerprint: pipeline:issue-triage:agent:execute-github-copilot-cli:failure
  • Details: Run #65 (issues event, created 2026-08-30T19:19Z) failed in job agent at the Execute GitHub Copilot CLI step. Other jobs in the run (pre_activation, pat_pool, activation, detection, safe_outputs, conclusion) succeeded.
  • Action: Review the agent execution logs for the triage workflow run.
  • Investigation: dispatched (see table below).

All 4 vally-shard failures above trace back to the same single scheduled run (#33298539883, run #8011), alongside the pre-existing evaluation:failure-rate:warning finding — see Correlation note in Trends.

Investigation Results

Deep investigations are dispatched for new critical/warning findings. The grooming workflow links results ~3 hours after this run.

Finding Severity Investigation First Seen Result
Evaluation failure rate across all branches exceeds 30% (critical) Critical Dispatched 2026-08-30 ⏳ Investigation dispatched — results arriving shortly...
Evaluation workflow failed on main: vally (dotnet-ai--claude-haiku-4.5) — Run vally evaluations Critical Dispatched 2026-08-30 ⏳ Investigation dispatched — results arriving shortly...
Evaluation workflow failed on main: vally (dotnet-blazor--claude-opus-5) — Run vally evaluations Critical Dispatched 2026-08-31 ⏳ Investigation dispatched — results arriving shortly...
Evaluation workflow failed on main: vally (dotnet-data--claude-opus-5) — Run vally evaluations Critical Dispatched 2026-08-31 ⏳ Investigation dispatched — results arriving shortly...

✅ Resolved Since Yesterday (4)

These were in yesterday's report but are no longer detected in the last 24h.

Evaluation failure rate across all branches exceeds 30% (critical)

Re-sampled with the current 24h window (7 runs across all branches/events): 1 failure / 3 completed success-or-failure runs = ~25% failure rate — now below the 30% critical threshold, but still above 15%, so this demotes to the pre-existing pipeline:evaluation:failure-rate:warning finding rather than disappearing entirely (see Existing Findings).

Evaluation workflow failed on main: vally (dotnet-ai--claude-haiku-4.5) — Run vally evaluations

No recurrence of this specific shard/model failure in the last 24h — different shards (dotnet-blazor, dotnet-data, dotnet-maui, dotnet-diag) failed instead in today's scheduled run.

skill-coverage workflow run cancelled on main (comment job)

No cancelled/timed-out runs detected on main in the last 24h window.

Orphan plugin: dotnet-test-migration not in marketplace.json

plugins/dotnet-test-migration is now listed in .github/plugin/marketplace.json (./plugins/dotnet-test-migration) — no longer orphaned.

Note: pipeline:evaluation:vally-(dotnet-maui--claude-haiku-4.5)... and dotnet-diag--mai-code-1-flash-picker from two runs ago were already dropped in the prior run and remain absent.

Existing Findings (4)

These have been present since before today. Sorted by age.

Warning — Orphan plugin: dotnet-experimental not in marketplace.json · first seen 2026-05-14 · 68 occurrences

Fingerprint: infra:orphan-plugin:dotnet-experimental Category: Infra · Severity: Warning

The plugin directory plugins/dotnet-experimental/ has a valid plugin.json (skills field resolves correctly) but is still not listed in .github/plugin/marketplace.json. ~109 days outstanding.

Suggested action: Either add dotnet-experimental to marketplace.json if ready for consumers, or remove the plugin directory if no longer needed.

Warning — Validate PAT Pool workflow failed: Build summary step · first seen 2026-08-05 · 9 occurrences

Fingerprint: pipeline:validate-pat-pool:validate-copilot-pat-pool:build-summary:failure Category: Pipeline · Severity: Warning

The validate-pat-pool.yml workflow failed again at the Build summary step (Run #85, all 10 individual PAT validation steps succeeded, only the summary step failed).

Suggested action: This recurring failure has now persisted for 9+ occurrences since 2026-08-05 — file a dedicated tracking issue given the sustained recurrence; the summary-generation logic itself likely has a bug independent of the PATs it validates.

Warning — Evaluation failure rate across all branches exceeds 15% · first seen 2026-08-27 · 3 occurrences

Fingerprint: pipeline:evaluation:failure-rate:warning Category: Pipeline · Severity: Warning

24h window (7 total runs, all branches/events): 1 failure, 3 successes, 2 skipped, 1 action_required. Failure rate = 1/(1+3) = 25%, above the 15% warning threshold (demoted from critical — see Resolved section). Event breakdown: 1 schedule failure (the same run driving the 4 new vally-shard findings above), 0 pull_request/workflow_dispatch failures.

Suggested action: Continue monitoring; if failure rate persists above 15% next run, treat as trending upward.

Critical — Eval avg run duration exceeds 55 min (critical threshold) · first seen 2026-08-28 · 2 occurrences

Fingerprint: resource:eval-duration:critical Category: Resource · Severity: Critical

Sampled the last 30 evaluation.yml runs on main; for the 30 completed (success/failure) runs within the last 14 days, average duration is ~76.5 min, above the 55-min critical threshold (60-min workflow timeout).

Suggested action: Investigate whether the evaluation matrix has grown or a subset of shards are consistently slow; consider increasing the timeout or splitting the matrix to reduce per-run duration. This likely correlates with the recurring scheduled-run failures above.

Trends (7-day)

Metric Today 7d Avg Δ Trend
Eval duration (min, main, n=30, 14d) ~76.5 ~93 (prior estimates) -16.5 ✅
Eval success rate (main, n=30 completed) 72% (13/18 strict) ~76% -4pp ⚠️
Eval success rate (all branches, 24h, n=3 completed) 75% (3/4) — wait, see note 50% (prior day) ~ ✅
Eval scheduled cancellation rate (24h, main) 0% (0/1) 0% 0 ➡️
Workflow failure rate (7d) not fully computed this run (time budget) — — ➡️
Compute hours/day not computed this run (time budget) — — ➡️

Correlation: All 4 new critical vally-shard failures (dotnet-blazor, dotnet-data, dotnet-maui, dotnet-diag) trace back to the same single scheduled run (#33298539883), consistent with last run's pattern where multiple shards failed together in one scheduled run. Combined with the still-critical eval-duration finding (~76.5 min avg) and the still-active evaluation:failure-rate:warning, this continues to suggest the eval pipeline occasionally fails broadly within a single scheduled run — worth checking whether resource contention, a shared dependency, or a transient infra issue around scheduled-run time is the common cause.

Recommendations:

  1. Investigate the single failed scheduled run (#33298539883 / run #8011) holistically — 4 shards failed together at "Run vally evaluations", suggesting a shared cause rather than per-shard flakiness.
  2. The eval-duration critical finding persists at 2 occurrences (~76.5 min avg) — prioritize investigating matrix growth or slow shards; this may also explain schedule-driven shard failures if runs are timing out under load.
  3. File a standalone tracking issue for the validate-pat-pool Build-summary failure, now at 9 occurrences since 2026-08-05.
  4. Resolve the remaining dotnet-experimental orphan-plugin gap in marketplace.json (109+ days outstanding) — dotnet-test-migration was resolved this run, showing the pattern is fixable.
  5. Investigate the new Issue Triage workflow failure (Execute GitHub Copilot CLI step) — first occurrence, could indicate an emerging Copilot CLI/agent issue worth tracking if it recurs.

⚠️ Skipped Pages deployment check (I5): no Pages-related MCP tool/endpoint available this run — check appears not applicable or not reachable with available tooling. ⚠️ Skipped relaxed-skill-validation (I3): no validate-skills.yml file exists in .github/workflows/; closest analog skill-validator.yml does not contain fail-on-warning: false — no finding raised. ⚠️ Skipped verdict-warn-only (I4): evaluation.yml does not contain --verdict-warn-only — no finding raised. ⚠️ I6 (unpinned third-party actions): scanned all workflow YAML files for non-actions/* references — found github/codeql-action and super-linter/super-linter, both already pinned to full commit SHAs. No unpinned-action findings. ⚠️ I7 (orphan skills): all 15 plugin directories under plugins/*/ have a plugin.json at their root resolving a ./skills/ path containing the discovered skill directories; no orphan skills detected. ⚠️ U3 (cost trend) and full 7-day workflow-failure-rate/compute-hours trends not computed this run due to time budget — carried over as "not computed" from prior runs.

Generated by DevOps Health Check agentic workflow · Run #33353465682 · 2026-08-31T03:20 UTC> Generated by DevOps Daily Health Check · auto · 185.9 AIC · ⌖ 4.32 AIC · ⊞ 20.5K · ◷

Daily Health Check — 2026-09-12

Status: 16 critical · 4 warnings · 0 info Since yesterday: 17 new · ✅ 6 resolved · 3 unchanged

Maintainer action needed: please pin this issue as the canonical health dashboard and unpin/close any stale duplicate.

New Findings (17)

These appeared since the last health check (2026-08-31).

Evaluation failure rate across all branches exceeds 30%

  • Fingerprint: pipeline:evaluation:failure-rate:critical
  • Details: 43 evaluation runs in the last 24 hours: 11 failures, 10 successes, 1 cancellation, 19 skipped, and 2 other conclusions. Failure rate is 52.4% (11/21, cancellations excluded); non-success rate is 30.2% (13/43). Failures comprised 7 workflow-dispatch, 3 pull-request-review, and 1 scheduled run. Recent samples: run 9098, run 9095, run 9092, run 9085, and run 9082.
  • Common failures: Copilot token selection dominated the failed jobs; session-data publishing also failed repeatedly.
  • Action: Investigate token-pool availability first, then rerun affected evaluations.
  • Investigation: dispatched.

Evaluation failed: csharp-refactoring token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet--csharp-refactoring--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: The same job failed at Select available Copilot token from pool in three main-branch dispatches: 9095, 9085, and 9080.
  • Action: Check PAT pool exhaustion or selection logic before retrying PR #873 evaluations.
  • Investigation: dispatched.

Evaluation failed: publish session data

  • Fingerprint: `pipeline:evaluation:publish-session-(redacted)
  • Details: publish-session-data failed at Push to dashboard-session-data branch (dotnet/skills-data) in the same three main-branch dispatches: 9095, 9085, and 9080.
  • Action: Confirm whether this is downstream fallout from missing evaluation artifacts or an independent branch-push failure.

Scheduled evaluation failed: dotnet-advanced token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-advanced--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-ai token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-ai--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-aspnetcore token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-aspnetcore--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-blazor token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-blazor--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-data token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-data--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-diag token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-diag--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-maui token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-maui--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-msbuild default token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-msbuild--shard-default--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-msbuild heavy token selection

  • Fingerprint: pipeline:evaluation:evaluate-/-vally-(dotnet-msbuild--shard-heavy--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
  • Details: Scheduled run 9057 failed before evaluation at token selection.
  • Action: Restore token-pool capacity and retry the shard.

Scheduled evaluation failed: dotnet-msbuild medium token selection

  • Fingerprint: `pipeline:evaluation:evaluate-/-vally-(dotnet-msbuild--shard-medium--claude-sonnet-4.6):select-available-copilot-token-from-poo

Source: dotnet/skills

View original on GitHubView discussion on GitHub