run the docker image on Mac with mps
Author: eslam-gomaaCreated Oct 18, 2023Updated Aug 7, 2025
Labelstype/question
Hi, Is it possible to run the docker image on Mac M1 with GPU mode ?
the docker command I'm using
export CUDA_VISIBLE_DEVICES=0
docker run \
-p 7860:7860 \
--shm-size=10.24gb \
--rm --init \
-e CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES \
-v /etc/passwd:/etc/passwd:ro \
-v /etc/group:/etc/group:ro \
-u 0:0 \
-v "${HOME}"/h2ogpt/.cache:/workspace/.cache \
-v "${HOME}"/h2ogpt/save:/workspace/save \
-v "${HOME}"/h2ogpt/user_path:/workspace/user_path \
-v "${HOME}"/h2ogpt/db_dir_UserData:/workspace/db_dir_UserData \
gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0 /workspace/generate.py \
--base_model=h2oai/h2ogpt-4096-llama2-7b-chat \
--use_safetensors=True \
--prompt_type=llama2 \
--save_dir='/workspace/save/' \
--use_gpu_id=False \
--score_model=None \
--max_max_new_tokens=2048 \
--max_new_tokens=1024 \
--top_k=5 \
--top_k_docs=5 \
--llamacpp_dict="{'n_gpu_layers':0,'n_batch':128}" \
--langchain_mode=UserDataSource: h2oai/h2ogpt