How to OPT model with PowerInfer?

Author: wuooo339Created Dec 24, 2024Updated Jul 23, 2025
Labelsquestion

Prerequisites

Before submitting your question, please ensure the following:

  • I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
  • I have carefully read and followed the instructions in the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).

Question Details

I have read your article of PowerInfer and I have seen you use OPT-30B to compare with llama.cpp.But I can not read any information about OPT in this READMD?

Additional Context

I want to test OPT model using PowerInfer.So I might need your help? I am from Harbin Institude of Technology studying HPC(high performance computing).And I am trying different offloading strategies recently.