Open-O1

Open O1: A Model Matching Proprietary Power with Open-Source Innovation

Homepage：OpenO1 Huggingface: OpenO1 Discord：OpenO1-Discord Slack：OpenO1-Slack

Mission and Vision for future

Our Open O1 aims to match the powerful capabilities of the proprietary OpenAI O1 model, empowering the community with advanced open-source alternatives. Our model has been developed by curating a set SFT data for CoT Activation, which was then used to train both LLaMA and Qwen models. This training approach has endowed the smaller models with enhanced long-reasoning and problem-solving capabilities.

As the Open O1 project progresses, we will continue to push the boundaries of what's possible with large language models. Our vision is to create a model that not only achieves O1-like performance but also leads the way in test-time scaling, making advanced AI capabilities accessible to all. Through community-driven development and a commitment to ethical practices, Open O1 will be a cornerstone in the advancement of AI, ensuring that the future of technology is open and beneficial to all.

Demo

Welcome to experience our model! Note: The model deployed on HuggingFace may have undergone quantization, which can lead to some performance degradation. If you'd like to experience the original performance of the model, you're encouraged to deploy it on your own infrastructure.

How many letters 'r' in the word 'strawberry'?

Input

How many letters 'r' in the word 'strawberry'?

…

Output

There are three letters 'r' in the word 'strawberry'.

News

[11/21/2024] OpenO1 SFT data for CoT Activation release！Welcome to download on OpenO1-SFT data

[10/09/2024] OpenO1 model-v0.1 release！

[10/05/2024] OpenO1 initial release！

Content Navigation

Model Deployment & Chat templates

Model Deployment

To quickly get started with Open O1, we provide some steps for you to enjoy our model.

Model Download

• OpenO1-LLama-8B-v0.1
• OpenO1-Qwen-7B-v0.1

Deployment

1. Git clone this repository.

git clone https://github.com/OpenSource-O1/Open-O1.git

1. Install the dependencies listed in Deployment/requirements.txt.

2. Execute the script below.

python Deployment/app.py

1. You can choose OpenO1-Qwen-7B-v0.1 or OpenO1-LLama-8B-v0.1 model, which can be obtained in huggingface.

Chat templates

The chat templates of Open O1 is following LLaMA3.1's. The details can be obtained in chat-templates.

dialog = [
    {"role": "user", "content": "What's the weather like today?"},
    {"role": "assistant", "content": "It's sunny and warm, around 75°F."}
]

After using chat templates to process, the converted result is as follows：

user
What's the weather like today?assistant
It's sunny and warm, around 75°F.assistant

System Performance

The following table provides a comprehensive comparison of the performance between llama3.1-8b-instruct and our model across multiple benchmarks. These evaluations were conducted in a zero-shot setting, meaning the models were tested without task-specific fine-tuning, highlighting their ability to generalize across diverse tasks. These benchmarks assess various aspects of reasoning, knowledge, and understanding in different domains, offering a clear indication of how each model handles complex tasks without prior exposure or specific task-related training. Our model consistently demonstrates competitive or superior performance, showcasing advancements in areas critical to reasoning, mathematical understanding, and general AI capabilities.

• GSM8K: Our model outperforms llama3.1-8b-instruct with a score of 85.82, demonstrating better reasoning ability in math word problems.
• MATH: It's important to note that the official score for llama3.1-8b-instruct on MATH is 51.9, but this was achieved in a CoT (Chain of Thought) setting. In our evaluation, we reproduced the result in a zero-shot setting, where llama3.1-8b-instruct scored lower at 47.42, while our model achieved 52.88, showing a significant improvement.
• MMLU: Our model leads with 70.45, indicating stronger general knowledge and understanding.
• Hellaswag: llama3.1-8b-instruct scores 68.43, slightly ahead of our model at 67.77.
• ARC-C: In ARC-C, our model reaches 86.52, outperforming llama3.1-8b-instruct.
• BBH: Our model achieves 58.43, surpassing llama3.1-8b-instruct’s score of 53.64.

The results highlights our model's superior performance in most benchmarks, with notable improvements in MATH, MMLU, ARC-C, and BBH.

Available Models

• OpenO1-Qwen-7B-v0.1
• OpenO1-LLama-8B-v0.1

Training Details

The training process for Open O1 utilizes the configuration settings from Llama Factory to optimize performance. This section includes details on the datasets used, training methodologies, and relevant hyperparameters.

method

stage: sft
do_train: true
finetuning_type: full
deepspeed: ds_z3_config.json

dataset

dataset: o1_style_dataset
template: llama3/qwen
cutoff_len: 4096
overwrite_cache: true
preprocessing_num_workers: 16

output

logging_steps: 10
save_steps: 1000
plot_loss: true
overwrite_output_dir: true

train

per_device_train_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 1.0e-5
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000

eval

val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 200

❓FAQ

Summarizing and updating...

⚠️Limitations

Open O1 is currently in its early stages of development. Open O1 primarily exhibits o1-like reasoning characteristics and broad search thinking capabilities. However, there is still significant progress to be made before it fully achieves O1 capabilities.

⭐Star History

Reference

• Learning to Reason with LLMs – A detailed blog post by OpenAI discussing methods to enhance reasoning abilities in large language models.

• OpenAI O1 Mini: Advancing Cost-Efficient Reasoning – An OpenAI blog that introduces the O1 Mini model and explores its efficiency in reasoning tasks.

• Awesome-LLM-Strawberry – A curated list of resources and tools related to large language models (LLMs) and reasoning capabilities, including O1.

✍Example Demonstrations

Reasoning Case

• Reasoning Case1
• Reasoning Case2

Coding

• Coding Case1
• Coding Case2

Math

• Math Case1
• Math Case2

Physics

• Physics Case1
• Physics Case2

Cipher

• Cipher Case1
• Cipher Case2

Counterfactual

• Counterfactual Case1
• Counterfactual Case2

Data Analysis

• Data Analysis Case1
• Data Analysis Case2

Administrative Aptitude Test(AAT)

• AAT Case1
• AAT Case2

Puzzle

• Puzzle Case1
• Puzzle Case2

Logic

• Logic Case1
• Logic Case2

Future Todo

Disclaimer

We have not yet reached O1 level, but we are on the way. The first version has shown performance improvements on LLaMA and Qwen, with some promising data, but the style is still very close to O1. This project is non-profit. We welcome the entire community to join us in this journey. Moving forward, we will open source all the code and materials from the development process.

Citation

If you find our model, data, code useful, welcome to cite our paper

@article{
    supplement,
    title={},
    author={OpenO1 Team},
    journal={},
    url={},
    year={}
}

Acknowledgements(Updating)

This repo benefits from LLaMA-Factory, OpenRLHF, awesome-o1. Thanks for their wonderful and solid works.

Feedback

If you have any questions, please submit them through GitHub Issues.

• Before doing so, we encourage you to review the FAQ section to see if your question has already been addressed, and check previous issues for any relevant discussions.
• Please kindly use our dedicated issue template for submitting.
• Appreciate your politeness and cooperation in fostering a positive and collaborative community.

Appendix

Reasoning Case1

Input

strawberry里有几个r？

Thought

Alright, I need to determine how many 'r's are in the word "strawberry." Let me start by carefully examining each letter in the word to identify all instances of the letter 'r'.

First, I'll list out the letters of