百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
L

llamacpp-python

> 编程语言
开源

Python 与 LLaMA.cpp 的绑定

199 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

Python 与 LLaMA.cpp 的绑定

Python bindings for llama.cpp

Important

  • The Python API has changed significantly in the recent weeks and as a result, I have not had a chance to update cli.py or chat.py to reflect the new changes. The scripts under examples/simple.py and examples/simple_low_level.py should give you an idea of how to use the library.

Install

From PyPI

bash
pip install llamacpp

Build from Source

bash
pip install .

Get the model weights

You will need to obtain the weights for LLaMA yourself. There are a few torrents floating around as well as some huggingface repositories (e.g https://huggingface.co/nyanko7/LLaMA-7B/). Once you have them, copy them into the models folder.

ls ./models
65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model

Convert the weights to GGML format using llamacpp-convert. Then use llamacpp-quantize to quantize them into INT4. For example, for the 7B parameter model, run

llamacpp-convert ./models/7B/ 1
llamacpp-quantize ./models/7B/
llamacpp-cli

Note that running llamacpp-convert requires torch, sentencepiece and numpy to be installed. These packages are not installed by default when your install llamacpp.

Command line interface

The package installs the command line entry point llamacpp-cli that points to llamacpp/cli.py and should provide about the same functionality as the main program in the original C++ repository. There is also an experimental llamacpp-chat that is supposed to bring up a chat interface but this is not working correctly yet.

API

Documentation is TBD. But the long and short of it is that there are two interfaces

  • LlamaInference - this one is a high level interface that tries to take care of most things for you. The demo script below uses this.
  • LlamaContext - this is a low level interface to the underlying llama.cpp API. You can use this similar to how the main example in llama.cpp does uses the C API. This is a rough implementation and currently untested except for compiling successfully.

Demo script

See llamacpp/cli.py for a detailed example. The simplest demo would be something like the following:

…

ToDo

  • Investigate using dynamic versions using setuptools-scm (Example: https://github.com/pypa/setuptools_scm/blob/main/scm_hack_build_backend.py)

Issues· 0 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

C++

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月18日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言