evals · Issues· 340 open

[Architecture Proposal] Hardware-Gated Containment Framework (Genesis Protocol V5.0) for Recursive Self-Improvement

Proposal: harness-level failure attribution in eval reports (model vs harness vs eval definition)

Proposal: detect tool-trajectory regressions before deployment

Proposal: RES actor-indexing behavioral eval (20 samples, no custom code)

False-positive "distillation" ban on $200/mo Pro subscriber — automated appeal rejected by bot, requesting human review from Trust & Safety

Proposal: deterministic eval for agent action-boundary violations

Proposal: Shared Sparse Latent Neural Bus for Local ↔ Cloud Models

HttpRecorder silently discards logged eval events; HTTP error responses never trigger the fallback at any failure rate

API retry helper retries forever (no max_tries/max_time) and the error-detection branch in its callers is dead code

Model-graded classify: unparseable judge output silently becomes minimum score

[Bug] Context/Token Explosion in Geometric Script Generation (CadQuery/CalculiX)

LangChain completion wrappers drop runtime kwargs

Steganography does not flag missing required protocol fields

SchellingPoint mutates process-wide random state

MakeMeSay get_content fails on its documented dict response type