百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
D

DEER

> 编程语言
开源

这是 DEER 的仓库,DEER 是一种用于大型推理语言模型的动态早期退出推理方法。

200 stars0 点赞0 次浏览
访问官网GitHub

工具介绍

这是 DEER 的仓库,DEER 是一种用于大型推理语言模型的动态早期退出推理方法。

DEER : Dynamic Early Exit in Reasoning Models

This is the repository of our paper: Dynamic Early Exit in Reasoning Models.

DEER monitors model behavior at potential reasoning transition points and dynamically terminates the next reasoning chain’s generation when the model exhibits high confidence in a trial answer. It is consistently effective on 11 cutting-edge reasoning LLMs of varying series and sizes, reducing the length of CoT sequences by an average of 19.1% - 80.1% while improving accuracy by 0.3% - 5.0%.


Latest Updates

  • [2025/05/20] Released DEER code for mathematical reasoning tasks (HuggingFace & vLLM).
  • [Coming Soon] DEER for code generation tasks & Branch-Parallel Decoding Acceleration.

Key Results

Results on 11 reasoning models with 16k token budgets. "Acc" denotes accuracy, "Tok" denotes token count, and "CR" denotes compression rate.

Experimental results presented in bar charts.


Quick Start

1. Installation

bash
git clone https://github.com/yourusername/DEER.git
cd DEER
pip install -r requirements.txt

2. DEER on vLLM (Recommended)

Considering efficiency, we recommend reproducing the results using the code based on the vLLM framework.

For Most Reasoning Models

bash
CUDA_VISIBLE_DEVICES=1 python ../vllm-deer.py \
    --model_name_or_path "./DeepSeek-R1-Distill-Qwen-14B" \
    --dataset_dir "./data/" \
    --output_path "./outputs" \
    --dataset "math" \
    --threshold 0.95 \
    --max_generated_tokens 16000 \
    --think_ratio 0.6 \
    --batch_size 2000 \
    --policy avg1 \
    --dtype bfloat16 \
    --gpu-memory-utilization 0.9 \ 

or run:

bash
bash ./bashes/bash-vllm-deer.sh.

For Qwen3 Models

bash
CUDA_VISIBLE_DEVICES=1 python ../vllm-deer-qwen3.py \
    --model_name_or_path "./Qwen3-4B" \
    --dataset_dir "./data/" \
    --output_path "./outputs" \
    --dataset "math" \
    --threshold 0.95 \
    --max_generated_tokens 16000 \
    --think_ratio 0.8 \
    --batch_size 2000 \
    --dtype bfloat16 \
    --policy avg2 \
    --gpu-memory-utilization 0.9 \

or run:

bash
bash ./bashes/bash-vllm-deer-qwen3.sh.

In our experiments, we found that Qwen3-series models tend to be over-confident in confidence prediction, so we made some modifications to its implementation.

  • The calculation of answer confidence was changed from arithmetic mean to geometric mean.
  • An additional condition must be satisfied for early exit: the model must generate after the trial answer.

3. DEER on Transformers

For inference using HuggingFace Transformers (without vLLM), run:

bash
bash ./bashes/bash-vanilla-deer.sh

Evaluation

DEER currently supports evaluation on 7 reasoning benchmarks. The rule-based evaluation for these benchmarks is based on the code implementation from the project LIMO.

bash
python ../check.py \
    --model_name_or_path "./DeepSeek-R1-Distill-Qwen-14B" \
    --data_name "math" \
    --generation_path "your_output.jsonl" \

or run

bash
bash ./bashes/bash-check-correct.sh

Citation

If you use DEER in your research, please cite our paper:

bibtex
@article{yang2025dynamic,
  title={Dynamic Early Exit in Reasoning Models},
  author={Yang, Chenxu and Si, Qingyi and Duan, Yongjie and Zhu, Zheliang and Zhu, Chenyu and Lin, Zheng and Cao, Li and Wang, Weiping},
  journal={arXiv preprint arXiv:2504.15895},
  year={2025}
}

Community

Join our WeChat group for discussions:

Issues· 7 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Python

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言