How to OPT model with PowerInfer?
Author: wuooo339Created Dec 24, 2024Updated Jul 23, 2025
Labelsquestion
Prerequisites
Before submitting your question, please ensure the following:
- I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- I have carefully read and followed the instructions in the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Question Details
I have read your article of PowerInfer and I have seen you use OPT-30B to compare with llama.cpp.But I can not read any information about OPT in this READMD?
Additional Context
I want to test OPT model using PowerInfer.So I might need your help? I am from Harbin Institude of Technology studying HPC(high performance computing).And I am trying different offloading strategies recently.
Source: Tiiny-AI/PowerInfer