opencompass · Issues· 404 open

`gsm8k_postprocess` breaks numeric answers at thousands separators (`\boxed{$9{,}500}` → `500`, `\boxed{8,000}` → `000`)

TopkRetriever crashes on transformers 5.x: tokenizer.encode_plus was removed

Grading integrity: model code rewrites ground truth in three benchmarks; CRUXEval splices prediction into assertion; unparseable verdicts exit denominator; LCEvaluator eval() on raw prediction (follow-up to #2535)

[Bug] ToxicEvaluator: expected_max_toxicity uses built-in max() on a NaN-containing array, so the metric depends on sample order

OpenAI-compat: OpenAISDK openai_api_base /v1, not the requests OpenAI full-path default

Feature request: EvalPort import/export for CustomDataset (adapter already built and tested)

[Feature] DIK-structured Turkish syntactic dependency annotation dataset for structured linguistic evaluation

CompassAcademic: 4 of 7 objective boards have an unseparated #1, two are exact ties broken by display order

[Bug] Qwen3.5 cannot run when using HuggingFacewithChatTemplate

MedFailBench integration proposal for clinical AI safety evaluation

[Bug] TypeError: Expected a datasets.Dataset or a datasets.DatasetDict object, but got {'train': <modelscope.msdatasets.ms_dataset.MsDataset object at xxx>, 'test': <modelscope.msdatasets.ms_dataset.MsDataset object at xxx>}

[Feature] 目前是否有适配Codeforces、SWE Verified、Aider-Polyglot这些在R1中出现的数据集的计划呢？

[Feature] 请问主观评测脚本支持用本地模型作为judge模型吗？

[Bug] 在对DeepSeek-R1-Distill-Qwen-1.5B模型评测livecodebench数据集时，lcb_test_output为什么为0呢？

[Feature] Add another OpenAISDK model using prompt completion API