Installation Steps

Sorting out the installation steps:

precondition:

download and install node.js from https://nodejs.org/en add "%NODE_PATH%","C:\Program Files\nodejs" & "C:\Program Files\nodejs\node_global" to system environment variable (by default)

then install pnpm:

npm install -g pnpm

installation:

clone to path: C:\Users\<your_username>:

cd C:\Users\<your_username>
git clone https://github.com/cumulo-autumn/StreamDiffusion.git
cd streamdiffusion

create python3.10 virtual environmen and activate it:

python -m venv streamdiffusion
streamdiffusion\scripts\activate.bat

install torch for cuda 11.8:

pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu118

Install StreamDiffusion (latest version, recommended):

pip install git+https://github.com/cumulo-autumn/StreamDiffusion.git@main#egg=streamdiffusion[tensorrt]

Install TensorRT extension:

python -m streamdiffusion.tools.install-tensorrt

examples:

screen function: installation: navigate to examples folder and install requirements:

cd examples
pip install -r screen/requirements.txt

measure the performance of streamdiffusion

python benchmark/multi.py

If everything goes well press Ctrl+C to exit the current running. run:

python screen/main.py

If everything goes well press Ctrl+C to exit the current running.

img2img function (place source images in path" ./images/input/", current path: C:\Users<user name>\StreamDiffusion\examples): Image-to-image for a single image:

python img2img/single.py --input ../images/inputs/input.png --output ../images/outputs/output.png

If everything goes well press Ctrl+C to exit the current running. Image-to-image for multiple images:

python img2img/multi.py --input ../images/inputs --output-dir ../images/outputs

If everything goes well press Ctrl+C to exit the current running.

txt2img (current path: C:\Users\\StreamDiffusion\examples): text-to-image for a single image:

python txt2img/single.py --output ../images/outputs/output.png --prompt "<your prompts>"

If everything goes well press Ctrl+C to exit the current running. text-to-image for multiple images:

python txt2img/multi.py --output ../images/outputs --prompt "<your prompts>"

If everything goes well press Ctrl+C to exit the current running.

vid2vid: install extra dependencies (current path: C:\Users\\StreamDiffusion\examples):

pip install -r vid2vid/requirements.txt

run (place source video in path" ./images/input/"):

python vid2vid/main.py --input ..images/inputs/input.mp4 --output ../images/outputs/output.mp4

If everything goes well press Ctrl+C to exit the current running.

other command line options:

1. --model_id_or_path: --model_id_or_path: "" #hugging face --model_id_or_path: "<model path/model name>" #local, recommend
2. --lora_dict: --lora_dict "{'<lora_1 path>' : <lora_1 scale> , '<lora_2 path>' : <lora_2 scale> }"
3. --prompt: --prompt ""
4. --negative prompts: --negative_prompts ""

demo:

realtime-img2img: every single time to execute "realtime-img2img" function you have to run every line below from "npm i"

cd C:\Users\<user name>\StreamDiffusion\demo\realtime-img2img\frontend
npm i
npm run build
cd..
pip install -r requirements.txt

run (need long time for launch till webpage pop up automatically at "http://0.0.0.0:7860/"; there will be various errors messages just disregard it. see issues #130 for details):

python main.py  --acceleration tensorrt

If everything goes well press Ctrl+C to exit the current running.

realtime-txt2img:

cd C:\Users\<user name>\StreamDiffusion\demo\realtime-txt2img
pip install -r requirements.txt
cd frontend
pnpm i
pnpm run build
cd..

run:

python main.py

If everything goes well press Ctrl+C to exit the current running.

vid2vid:

cd C:\Users\<user name>\StreamDiffusion\demo\vid2vid
pip install -r requirements.txt

run:

python app.py

If everything goes well press Ctrl+C to exit the current running.