A simple toy demo of a local voice assistant with whisper and large language model.
A simple toy demo of a local voice assistant with whisper and large language model.
English | 简体中文
This is a simple Python script project that allows dialogue with a local large language model through voice.
The voice recognition part of this project is from the Apple MLX example repo, and the textual responses are generated using the Yi model from 01.AI. For more details, see the [Acknowledgments](## Acknowledgments) section.
├───main.py
├───models
├───prompts
├───recordings
├───tools
│ └───list_microphones.py
├───whisper
This project is a single-script project, with main.py containing all program logic. The models/ folder stores model files. prompts/ contains prompt words. recordings/ holds temporary recordings. tools/list_microphones.py is a simple script to view the microphone list, used in main.py to specify the microphone number. whisper/ is from the Apple MLX example repo, used for recognizing user's voice input.
This project is based on the Python programming language, and the Python version used for program operation is 3.11.5. It is recommended to configure the Python environment using Anaconda. The following setup process has been tested and passed on macOS systems. Windows and Linux can use speech_recognition and pyttsx3 to replace the whisper and say commands mentioned below. The following are console/terminal/shell commands.
conda create -n VoiceAI python=3.11
conda activate VoiceAI
pip install -r requirements.txt
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python
# Install audio processing tools
brew install portaudio
pip install pyaudio
The model files are stored in the models/ folder and specified in the script via the MODEL_PATH variable.
It is recommended to download the gguf format models from TheBloke and XeIaso, where the 6B model has a smaller memory footprint:
models/whisper-large-v3/, specified in the script via WHISP_PATH. The version converted by mlx-community can be directly downloaded.The voice recognition part of this project is based on OpenAI's whisper model, its implementation comes from the Apple MLX example repo. The version used in this project is from January 2024, #80d1867. In the future, users can fetch new versions as needed.
The responses in this project are generated by the large language model Yi from 01.AI, where Yi-34B-Chat is more powerful. The 8-bit quantized version made by TheBloke has a memory footprint of 39.04 GB and is recommended for use if hardware conditions permit. This model runs locally based on the LangChain framework and llama.cpp by Georgi Gerganov.
Thank you to all selfless programmers for their contributions to the open-source community!
No open issues yet, or sync has not completed.