open-r1

工具介绍

全面公开复制DeepSeek-R1

核心特点

• •Generate data from a smol distilled R1 model
• •Generate data from DeepSeek-R1
• •src/open_r1: contains the scripts to train models as well as generate synthetic data:
• •grpo.py: trains a model with GRPO on a given dataset.
• •sft.py: performs a simple SFT of a model on a dataset.
• •generate.py: generates synthetic data from a model using Distilabel.
• •Makefile: contains easy-to-run commands for each step in the R1 pipeline leveraging the scripts above.
• •Step 1: replicate the R1-Distill models by distilling a high-quality corpus from DeepSeek-R1.
• •Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will likely involve curating new, large-scale datasets for math, reasoning, and code.
• •Step 3: show we can go from base model to RL-tuned via multi-stage training.