[FEATURE REQUEST] Support for NPU hardware acceleration
Author: Flimsy-FoxCreated May 3, 2024Updated Jun 4, 2024
With both Intel and AMD releasing CPUs with NPUs (Neural Processing Unit) in their laptop CPU designs, NPUs being a part of the Orange Pi design, and the Raspberry Pi Project launching an official NPU HAT, it's clear that the AI space is moving heavily into hardware acceleration even for low-power systems.
This feature request is for NPU hardware acceleration support.
Source: getumbrel/llama-gpt