#970·h2ogpt

run the docker image on Mac with mps

Author: eslam-gomaaCreated Oct 18, 2023Updated Aug 7, 2025
Labelstype/question

Hi, Is it possible to run the docker image on Mac M1 with GPU mode ?

the docker command I'm using

bash
export CUDA_VISIBLE_DEVICES=0
docker run \
       -p 7860:7860 \
      --shm-size=10.24gb \
       --rm --init \
       -e CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES \
       -v /etc/passwd:/etc/passwd:ro \
       -v /etc/group:/etc/group:ro \
       -u 0:0 \
       -v "${HOME}"/h2ogpt/.cache:/workspace/.cache \
       -v "${HOME}"/h2ogpt/save:/workspace/save \
       -v "${HOME}"/h2ogpt/user_path:/workspace/user_path \
       -v "${HOME}"/h2ogpt/db_dir_UserData:/workspace/db_dir_UserData \
       gcr.io/vorvan/h2oai/h2ogpt-runtime:0.1.0 /workspace/generate.py \
          --base_model=h2oai/h2ogpt-4096-llama2-7b-chat \
          --use_safetensors=True \
          --prompt_type=llama2 \
          --save_dir='/workspace/save/' \
          --use_gpu_id=False \
          --score_model=None \
          --max_max_new_tokens=2048 \
          --max_new_tokens=1024 \
          --top_k=5 \
          --top_k_docs=5 \
          --llamacpp_dict="{'n_gpu_layers':0,'n_batch':128}" \
          --langchain_mode=UserData