当我运行 "bash training_scripts/opt/single_gpu/run_1.3b.sh" 时,步骤 1 失败

the error: [2025-12-16 23:27:41,996] [WARNING] [runner.py:232:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2025-12-16 23:27:41,997] [INFO] [runner.py:630:main] cmd = /root/anaconda3/envs/deepspeed/bin/python3.12 -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMF19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None --log_level=info main.py --model_name_or_path /home/nbw/model/opt-1.3b/ --gradient_accumulation_steps 8 --lora_dim 128 --zero_stage 0 --enable_tensorboard --tensorboard_path ./output --deepspeed --output_dir ./output [2025-12-16 23:27:48,547] [INFO] [launch.py:162:main] WORLD INFO DICT: {'localhost': [0]} [2025-12-16 23:27:48,547] [INFO] [launch.py:168:main] nnodes=1, num_local_procs=1, node_rank=0 [2025-12-16 23:27:48,547] [INFO] [launch.py:179:main] global_rank_mapping=defaultdict(<class 'list'>, {'localhost': [0]}) [2025-12-16 23:27:48,548] [INFO] [launch.py:180:main] dist_world_size=1 [2025-12-16 23:27:48,548] [INFO] [launch.py:184:main] Setting CUDA_VISIBLE_DEVICES=0 [2025-12-16 23:27:48,549] [INFO] [launch.py:272:main] process 8596 spawned with command: ['/root/anaconda3/envs/deepspeed/bin/python3.12', '-u', 'main.py', '--local_rank=0', '--model_name_or_path', '/home/nbw/model/opt-1.3b/', '--gradient_accumulation_steps', '8', '--lora_dim', '128', '--zero_stage', '0', '--enable_tensorboard', '--tensorboard_path', './output', '--deepspeed', '--output_dir', './output'] [rank0]:[W1216 23:27:56.691535111 ProcessGroupNCCL.cpp:4715] [PG ID 0 PG GUID 0 Rank 0] using GPU 0 as device used by this process is currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. You can pecify device_id in init_process_group() to force use of a particular device. tokenizer initialize start load_hf_tokenizer model_name_or_path /home/nbw/model/opt-1.3b/ exist tokenizer initialize end model initialize start The module name (originally ) is not a valid Python identifier. Please rename the original module to avoid import issues. model initialize end dataset initialize start Creating prompt dataset ['/home/nbw/datasets/Dahoas/rm-static/'], reload=False Creating dataset Dahoas_rm_static for train_phase=1 size=15252 Creating dataset Dahoas_rm_static for train_phase=1 size=1021 dataset initialize end /root/anaconda3/envs/deepspeed/lib/python3.12/site-packages/torch/utils/cpp_extension.py:2356: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST']. warnings.warn( ***** Running training ***** ***** Evaluating Perplexity, Epoch 0/1 ***** [rank0]: Traceback (most recent call last): [rank0]: File "/home/nbw/train/main.py", line 447, in <module> [rank0]: main() [rank0]: File "/home/nbw/train/main.py", line 397, in main [rank0]: Perplexity, eval_loss = evaluation(model, eval_dataloader) [rank0]: …