空代 / 在 HumanEval 上的重现失败率为 40%

gen_outputs_dict = gen_model.generate( **gen_inputs, pad_token_id=gen_tokenizer.eos_token_id, max_new_tokens=NEW_TOKENS, return_dict_in_generate=True, do_sample=True, temperature=TEMP, top_p=0.95, top_k=0, stopping_criteria=construct_stopping_criteria( "code", STOP_SEQS, gen_tokenizer, gen_device ), ) The stop tokens I'm using are a subset of those found in the Codex paper: STOP_SEQS = ["\nclass", "\ndef"].