一个包含 LogitsProcessor 的集合,用于为特定任务自定义和增强 LLM 的行为。
Struggling to get LLMs to follow your instructions? LogitsProcessorZoo offers a zoo of tools to use LLMs for specific tasks, beyond just grammar enforcement!
pip install logits-processor-zoo
…
For the detailed examples in each framework, please have a look at lpz_examples directory.
When vLLM logits processors are configured through the HTTP API, processor kwargs come from the request body.
If a processor receives tokenizer as a string, the server will only load it when the tokenizer name is explicitly
allowlisted in the LOGITS_PROCESSOR_ZOO_VLLM_ALLOWED_TOKENIZERS environment variable. Tokenizers are loaded from
local files only, so the tokenizer must already be available in the server environment.
LOGITS_PROCESSOR_ZOO_VLLM_ALLOWED_TOKENIZERS=Qwen/Qwen2.5-1.5B-Instruct \
vllm serve Qwen/Qwen2.5-1.5B-Instruct \
--dtype auto \
--api-key lpz-test \
--logits-processor-pattern "logits_processor_zoo.vllm"
Multiple trusted tokenizer names can be separated with commas.
A logits processor that adjusts the likelihood of the end-of-sequence (EOS) token based on the length of the generated sequence, encouraging or discouraging shorter answers.
A logits processor which boosts or diminishes the likelihood of tokens present in the prompt (and optionally EOS token) to encourage the model to generate tokens similar to those seen in the prompt or vice versa.
A logits processor which forces LLMs to use the given phrase before they finalize their answers. Most common use cases can be providing references, thanking user with context etc.
A logits processor to answer multiple choice questions with one of the choices. A multiple choice question is like:
I am getting a lot of calls during the day. What is more important for me to consider when I buy a new phone?
0. Camera
1. Screen resolution
2. Operating System
3. Battery
The goal is to make LLM generate "3" as an answer.
A logits processor which triggers phrases when it encounters a given token or after a specified time. One common use case is to force writing python code just after thinking:
trigger_python = TriggerPhraseLogitsProcessor(phrase="\n```python", trigger_token_phrase="",
tokenizer=tokenizer, trigger_count=1, trigger_after=True)
A logits processor that mitigates hallucinated model outputs by enforcing a predefined fallback phrase when token confidence falls below a specified threshold.
A logits processor that enforces the end-of-sentence (EOS) token after a specified maximum time passes, optionally waiting for a new line or a full stop. Useful for controlling generation time and ensuring responses complete within time constraints.
暂无开放 Issues，或尚未同步最近议题。