Verified OSS loop: claim leases, evidence receipts, and independent exact-head review
Describe the feature
SWE-agent already has the core execution artifact for this problem: a trajectory produced inside a reproducible SWEEnv, with batch execution and SWE-bench evaluation support. What is missing for use on live open-source backlogs is a repository-aware contribution mode around those runs.
A maintainer should be able to point SWE-agent at a project-selected issue queue without multiple runs duplicating work or presenting an unverified patch as a finished contribution.
Proposed workflow:
- Read candidates only from repository-owned labels/milestones/project fields.
- Before launch, check existing PRs and create an expiring claim tied to issue, base SHA, config, environment, and run ID.
- Execute the issue in the normal isolated
SWEEnv. - Emit a contribution receipt beside the trajectory:
- issue, base/head SHA, patch hash;
- SWE-agent config and environment setup;
- trajectory path/hash;
- pre-change reproduction when available;
- evaluation commands and results;
- model calls, tokens/cost, termination reason, and limitations.
- Run a separate reviewer trajectory against the exact candidate head. The reviewer does not edit it and returns
approve,changes required,duplicate, orblockedwith evidence. - Publish the candidate only through the repository's normal PR template and maintainer review process.
If the base revision, patch, or candidate head changes, the receipt and reviewer verdict become stale. A competing PR or changed maintainer direction should stop publication rather than create another implementation.
This would let SWE-agent move from “generate a patch for one issue” to “participate safely in a live project queue” without becoming the project's roadmap or merge authority.
Potential Solutions
Minimal local implementation
- Add an optional
contributewrapper around the existing single-issue runner. - Accept a repository-specific candidate command or GitHub query instead of defining universal priorities.
- Write
contribution_receipt.jsonnext to the normal.trajand log files. - Add a read-only reviewer configuration that receives issue text, repository policy, candidate diff, trajectory summary, and evaluation output.
- Store claim state in the issue or a GitHub check/comment with a lease expiry.
- Demonstrate on one real repository issue plus one duplicate-claim case.
SWE-agent-specific acceptance checks
- The recorded environment can be recreated from the receipt.
- Replaying evaluation against the exact patch produces the same result.
- The trajectory inspector can open both implementation and review trajectories.
- SWE-bench-style
PASS_TO_PASSandFAIL_TO_PASSdata is used when available, while non-SWE-bench repositories can supply project commands. - Cost/call limits and termination reasons are visible to maintainers.
- No worker can merge, change issue priority, or suppress an existing PR.
Alternatives
- Batch mode alone: good for benchmark instances, but it does not coordinate claims or live PR overlap.
- Trajectory alone: rich execution evidence, but not a compact exact-head maintainer receipt.
- CI alone: evaluates the resulting patch but does not capture environment setup, attempted reasoning path, duplicate state, or why the run stopped.
- Centralized global backlog: rejected; each repository should choose its own issue-selection rules and authority.
Source: SWE-agent/SWE-agent