Proposal: Add context-benchmarking Skill and CI Example

Author: ArielILCreated Jan 20, 2026Updated Jun 18, 2026

I've been diving into the advanced-evaluation and context-compression skills and found them incredibly useful. I noticed there is currently a gap between designing these evaluations and operationalizing them in a production workflow.

I would like to propose adding a new Operational Skill and a corresponding Example to bridge this gap.

The Proposal:

  1. New Skill (skills/operational-skills/context-benchmarking): A guide on how to set up a benchmarking feedback loop. This would cover metrics (latency vs. recall), preventing regression in retrieval quality, and safeguards for automated testing.

  2. New Example (examples/automated-benchmarking-pipeline): A working GitHub Action workflow that implements the existing llm-as-judge logic.

Implementation Details:

  • The example pipeline will use a Python script to run a test set against an agent configuration.

  • Cost Safety: To ensure this doesn't spike API usage for forks or the main repo, the GitHub Action will be configured to run only on workflow_dispatch (manual trigger) or specific PR labels (e.g., run-benchmark).

Why this matters:

As the repo grows, users need a way to verify that new context compression strategies (like context-optimization) don't accidentally degrade information retrieval. This validates the "theory" of the other skills.

Would you be open to a PR for this?

Source: muratcankoylan/Agent-Skills-for-Context-Engineering