[Feature] 请问主观评测脚本支持用本地模型作为judge模型吗?

Author: KAKSISCreated Jun 18, 2025Updated Jul 15, 2026

Describe the feature

examples/eval_subjective.py 在这个文件中,我把judge_models改为了vllmwithchattemplate的形式,似乎并不能正常评测,alpaca eval的最终输出结果为空。 请问主观评测脚本支持用本地模型作为judge模型吗?

Will you implement it?

  • I would like to implement this feature and create a PR!

Source: open-compass/opencompass