++ 基准 & “ ++ ”

After reading the contents of Chapter 7 on Agent evaluation, I have been wondering about one question: how can we avoid the contamination of public benchmarks? Many LLM/Agent benchmarks are public, which makes it easy for others to reproduce and compare. However, in the long run, these data may be used as pre-training or post-training data for subsequent models, and the evaluation may gradually shift from "ability testing" to "memory testing." I have encountered this problem myself. I have constructed an Agent-oriented evaluation dataset in the field of physical modeling, and I don't know how to publish it. If it is open source, it will be convenient for others to use, but it will soon become a memory game; if it is not public, it will be difficult for others to reproduce and use. So I would like to ask the authors and everyone: how can we achieve the greatest balance between openness and contamination in benchmarks? I would also like to hear your thoughts on future evaluation methods. PS: One method I have thought of is to deploy the dataset on a server, allowing other people's Agents to conduct evaluations directly on the server, so that they cannot see the dataset itself. However, I am concerned that the Agents may still extract the test data during the evaluation process, which will eventually lead to leakage.