404 Not Found for Ollama Embedding API
Describe the bug I deployed ollama in windows 11, and config the embedding model as below:
type: embedder provider: litellm_embedder models:
- model: ollama/nomic-embed-text:lastest alias: default api_base: http://host.docker.internal:11434 timeout: 120
But I get error as:
File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1586, in wrapper_async raise e File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1437, in wrapper_async result = await original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 3577, in aembedding raise exception_type( ^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2301, in exception_type raise e File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2270, in exception_type raise APIConnectionError( litellm.exceptions.APIConnectionError: litellm.APIConnectionError: OllamaException - Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/embed'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
To Reproduce Steps to reproduce the behavior:
- Run ollama with default configurations
- pull the llm model and nomic-embed-text:latest
- Configurate WrenAI according to https://docs.getwren.ai/oss/ai_service/guide/custom_llm
- Start WrenAI
- See the error in container wrenai-wren-ai-service-1
Expected behavior The container wrenai-wren-ai-service starts properly
Screenshots
-- Following message says the ollama can be accessed in docker
/mnt/c/Users/plowa/.wrenai$ docker run --rm curlimages/curl -H 'Content-Type: application/json' http://host.docker.internal:11434/api/tags % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0{"models":[{"name":"nomic-embed-text:latest","model":"nomic -embed-text:latest","modified_at":"2025-08-20T13:59:38.0871564+08:00","size":274302450,"digest":"0a109f422b47e3a30ba2b10eca18548e944e8a23 073ee3f3e947efcf3c45e59f","details":{"parent_model":"","format":"gguf","family":"nomic-bert","families":["nomic-bert"],"parameter_size":" 137M","quantization_level":"F16"}},{"name":"gemma3:270m","model":"gemma3:270m","modified_at":"2025-08-20T12:40:54.0908435+08:00","size":2 91554930,"digest":"e7d36fb2c3b3293cfe56d55889867a064b3a2b22e98335f2e6e8a387e081d6be","details":{"parent_model":"","format":"gguf","family ":"gemma3","families":["gemma3"],"parameter_size":"268.10M","quantization_level":"Q8_0"}},{"name":"qwen3:4b","model":"qwen3:4b","modified _at":"2025-08-19T15:30:22.9632116+08:00","size":2497293918,"digest":"e55aed6fe643f9368b2f48f8aaa56ec787b75765da69f794c0a0c23bfe7c64b2","d etails":{"parent_model":"","format":"gguf","family":"qwen3","families":["qwen3"],"parameter_size":"4.0B","quantization_level":"Q4_K_M"}}, {"name":"qwen2.5:7b-instruct","model":"qwen2.5:7b-instruct","modified_at":"2025-07-26T20:48:29.3137526+08:00","size":4683087332,"digest": "845dbda0ea48ed749caafd9e6037047aa19acfcfd82e704d7ca97d631a0b697e","details":{"parent_model":"","format":"gguf","family":"qwen2","familie s":["qwen2"],"parameter_size":"7.6B","quantization_level":"Q4_K_M"}},{"name":"gemma3:4b","model":"gemma3:4b","modified_at":"2025-07-25T23 :08:29.2829488+08:00","size":3338801804,"digest":"a2af6cc3eb7fa8be8504abaf9b04e88f17a119ec3f04a3addf55f92841195f5a","details":{"parent_mo 100 1687 100 1687 0 0 198k 0 --:--:-- --:--:-- --:--:-- 205ke":"4.3B","quantization_level":"Q4_K_M"}}]}
/mnt/c/Users/plowa/.wrenai$ docker run --rm curlimages/curl -H 'Content-Type: application/json' -d '{"model": "nomic-embed-text:latest", "prompt": "Hello world"}' http://host.docker.internal:11434/api/embed % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 113 100 51 100 62 55 67 --:--:-- --:--:-- --:--:-- 122 {"model":"nomic-embed-text:latest","embeddings":[]}
--- But I get 404 not found Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 3562, in aembedding response = await init_response # type: ignore ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/handler.py", line 87, in ollama_aembeddings response = await litellm.module_level_aclient.post(url=api_base, json=data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 135, in async_wrapper result = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 324, in post raise e File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 280, in post response.raise_for_status() File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 763, in raise_for_status raise HTTPStatusError(message, request=request, response=self) httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/embed'
Desktop (please complete the following information):
- OS: [Windows11]
- Browser [edge]
Wren AI Information
- Version: [0.27.0]
Additional context Add any other context about the problem here.
Relevant log output
config.yaml
type: llm provider: litellm_llm models:
- api_base: http://host.docker.internal:11434/v1 alias: default model: ollama_chat/qwen3:4b timeout: 600 kwargs: n: 1 temperature: 0
type: embedder provider: litellm_embedder models:
- model: ollama/nomic-embed-text:lastest alias: default api_base: http://host.docker.internal:11434 timeout: 120
type: engine provider: wren_ui endpoint: http://wren-ui:3000
type: engine provider: wren_ibis endpoint: http://ibis-server:8000
type: document_store provider: qdrant location: http://qdrant:6333 embedding_model_dim: 768 timeout: 120 recreate_index: true
type: pipeline pipes:
- name: db_schema_indexing embedder: litellm_embedder.default document_store: qdrant
- name: historical_question_indexing embedder: litellm_embedder.default document_store: qdrant
- name: table_description_indexing embedder: litellm_embedder.default document_store: qdrant
- name: db_schema_retrieval llm: litellm_llm.default embedder: litellm_embedder.default document_store: qdrant
- name: historical_question_retrieval embedder: litellm_embedder.default document_store: qdrant
- name: sql_generation llm: litellm_llm.default engine: wren_ui document_store: qdrant
- name: sql_correction llm: litellm_llm.default engine: wren_ui document_store: qdrant
- name: followup_sql_generation llm: litellm_llm.default engine: wren_ui document_store: qdrant
- name: sql_answer llm: litellm_llm.default
- name: semantics_description llm: litellm_llm.default
- name: relationship_recommendation llm: litellm_llm.default engine: wren_ui
- name: question_recommendation llm: litellm_llm.default
- name: question_recommendation_db_schema_retrieval llm: litellm_llm.default embedder: litellm_embedder.default document_store: qdrant
- name: question_recommendation_sql_generation llm: litellm_llm.default engine: wren_ui document_store: qdrant
- name: intent_classification llm: litellm_llm.default embedder: litellm_embedder.default document_store: qdrant
- name: misleading_assistance llm: litellm_llm.default
- name: data_assistance llm: litellm_llm.default
- name: sql_pairs_indexing document_store: qdrant embedder: litellm_embedder.default
- name: sql_pairs_retrieval document_store: qdrant embedder: litellm_embedder.default llm: litellm_llm.default
- name: preprocess_sql_data llm: litellm_llm.default
- name: sql_executor engine: wren_ui
- name: chart_generation llm: litellm_llm.default
- name: chart_adjustment llm: litellm_llm.default
- name: user_guide_assistance llm: litellm_llm.default
- name: sql_question_generation llm: litellm_llm.default
- name: sql_generation_reasoning llm: litellm_llm.default
- name: followup_sql_generation_reasoning llm: litellm_llm.default
- name: sql_regeneration llm: litellm_llm.default engine: wren_ui
- name: instructions_indexing embedder: litellm_embedder.default document_store: qdrant
- name: instructions_retrieval embedder: litellm_embedder.default document_store: qdrant
- name: sql_functions_retrieval engine: wren_ibis document_store: qdrant
- name: project_meta_indexing document_store: qdrant
- name: sql_tables_extraction llm: litellm_llm.default
settings: doc_endpoint: https://docs.getwren.ai is_oss: true engine_timeout: 30 column_indexing_batch_size: 50 table_retrieval_size: 10 table_column_retrieval_size: 100 allow_intent_classification: true allow_sql_generation_reasoning: true allow_sql_functions_retrieval: true enable_column_pruning: false max_sql_correction_retries: 3 query_cache_maxsize: 1000 query_cache_ttl: 3600 langfuse_host: https://cloud.langfuse.com langfuse_enable: true logging_level: DEBUG development: false historical_question_retrieval_similarity_threshold: 0.9 sql_pairs_similarity_threshold: 0.7 sql_pairs_retrieval_max_size: 10 instructions_similarity_threshold: 0.7 instructions_top_k: 10
Starting Ibis Server... WREN_NUM_WORKERS is not set. Using default value of 2. Number of workers: 2 [2025-08-19 06:08:29 +0000] [7] [INFO] Starting gunicorn 23.0.0 [2025-08-19 06:08:29 +0000] [7] [INFO] Listening at: http://0.0.0.0:8000 (7) [2025-08-19 06:08:29 +0000] [7] [INFO] Using worker: app.worker.WrenUvicornWorker [2025-08-19 06:08:29 +0000] [8] [INFO] Booting worker with pid: 8 [2025-08-19 06:08:29 +0000] [9] [INFO] Booting worker with pid: 9 Starting Ibis Server... WREN_NUM_WORKERS is not set. Using default value of 2. Number of workers: 2 [2025-08-20 05:06:24 +0000] [7] [INFO] Starting gunicorn 23.0.0 [2025-08-20 05:06:24 +0000] [7] [INFO] Listening at: http://0.0.0.0:8000 (7) [2025-08-20 05:06:24 +0000] [7] [INFO] Using worker: app.worker.WrenUvicornWorker [2025-08-20 05:06:24 +0000] [8] [INFO] Booting worker with pid: 8 [2025-08-20 05:06:24 +0000] [9] [INFO] Booting worker with pid: 9 Starting Ibis Server... WREN_NUM_WORKERS is not set. Using default value of 2. Number of workers: 2 [2025-08-21 01:25:36 +0000] [6] [INFO] Starting gunicorn 23.0.0 [2025-08-21 01:25:36 +0000] [6] [INFO] Listening at: http://0.0.0.0:8000 (6) [2025-08-21 01:25:36 +0000] [6] [INFO] Using worker: app.worker.WrenUvicornWorker [2025-08-21 01:25:36 +0000] [7] [INFO] Booting worker with pid: 7 [2025-08-21 01:25:36 +0000] [8] [INFO] Booting worker with pid: 8 Starting Ibis Server... WREN_NUM_WORKERS is not set. Using default value of 2. Number of workers: 2 [2025-08-22 05:43:47 +0000] [7] [INFO] Starting gunicorn 23.0.0 [2025-08-22 05:43:47 +0000] [7] [INFO] Listening at: http://0.0.0.0:8000 (7) [2025-08-22 05:43:47 +0000] [7] [INFO] Using worker: app.worker.WrenUvicornWorker [2025-08-22 05:43:47 +0000] [8] [INFO] Booting worker with pid: 8 [2025-08-22 05:43:47 +0000] [9] [INFO] Booting worker with pid: 9 Starting Ibis Server... WREN_NUM_WORKERS is not set. Using default value of 2. Number of workers: 2 [2025-08-22 11:35:02 +0000] [7] [INFO] Starting gunicorn 23.0.0 [2025-08-22 11:35:02 +0000] [7] [INFO] Listening at: http://0.0.0.0:8000 (7) [2025-08-22 11:35:02 +0000] [7] [INFO] Using worker: app.worker.WrenUvicornWorker [2025-08-22 11:35:02 +0000] [8] [INFO] Booting worker with pid: 8 [2025-08-22 11:35:02 +0000] [9] [INFO] Booting worker with pid: 9
AI-SERVICE LOG
0%| | 0/1 [00:00<?, ?it/s]W0822 13:59:38.967 8 wren-ai-service:291] Calling QdrantDocumentStore.write_documents() with empty list INFO: 172.18.0.6:34744 - "GET /v1/semantics-preparations/f91a37d52b86f0e302421d752955d7a41f7509d1/status HTTP/1.1" 200 OK
100it [00:01, 77.18it/s]
100it [00:01, 77.15it/s]
embedding [src.pipelines.indexing.table_description.embedding()] encountered an error< Node inputs: {'chunk': "<Task finished name='Task-25' coro=<AsyncGraphAdap...", 'embedder': '<src.providers.embedder.litellm.AsyncDocumentEmbed...'}
Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 3562, in aembedding response = await init_response # type: ignore ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/handler.py", line 87, in ollama_aembeddings response = await litellm.module_level_aclient.post(url=api_base, json=data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 135, in async_wrapper result = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 324, in post raise e File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 280, in post response.raise_for_status() File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 763, in raise_for_status raise HTTPStatusError(message, request=request, response=self) httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/embed' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 122, in new_fn await fn(**fn_kwargs) if asyncio.iscoroutinefunction(fn) else fn(**fn_kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 219, in async_wrapper self._handle_exception(observation, e) File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 520, in _handle_exception raise e File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 217, in async_wrapper result = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/pipelines/indexing/table_description.py", line 97, in embedding return await embedder.run(documents=chunk["documents"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/backoff/_async.py", line 151, in retry ret = await target(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/providers/embedder/litellm.py", line 154, in run embeddings, meta = await self._embed_batch( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/providers/embedder/litellm.py", line 115, in _embed_batch responses = await asyncio.gather( ^^^^^^^^^^^^^^^^^^^^^ File "/src/providers/embedder/litellm.py", line 101, in embed_single_batch return await aembedding( ^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1586, in wrapper_async raise e File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1437, in wrapper_async result = await original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 3577, in aembedding raise exception_type( ^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2301, in exception_type raise e File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2270, in exception_type raise APIConnectionError( litellm.exceptions.APIConnectionError: litellm.APIConnectionError: OllamaException - Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/embed'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
Oh no an error! Need help with Hamilton? Join our slack and ask for help! https://join.slack.com/t/hamilton-opensource/shared_invite/zt-2niepkra8-DGKGf_tTYhXuJWBTXtIs4g
E0822 13:59:40.161 8 wren-ai-service:100] Failed to prepare semantics: litellm.APIConnectionError: OllamaException - Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/embed' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404 Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 3562, in aembedding response = await init_response # type: ignore ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/ollama/completion/handler.py", line 87, in ollama_aembeddings response = await litellm.module_level_aclient.post(url=api_base, json=data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 135, in async_wrapper result = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 324, in post raise e File "/app/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 280, in post response.raise_for_status() File "/app/.venv/lib/python3.12/site-packages/httpx/_models.py", line 763, in raise_for_status raise HTTPStatusError(message, request=request, response=self) httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://host.docker.internal:11434/api/embed' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/src/web/v1/services/semantics_preparation.py", line 92, in prepare_semantics await asyncio.gather(*tasks) File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 219, in async_wrapper self._handle_exception(observation, e) File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 520, in _handle_exception raise e File "/app/.venv/lib/python3.12/site-packages/langfuse/decorators/langfuse_decorator.py", line 217, in async_wrapper result = await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/src/pipelines/indexing/table_description.py", line 153, in run return await self._pipe.execute( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/hamilton/async_driver.py", line 375, in execute raise e File "/app/.venv/lib/python3.12/site-p
Source: Canner/WrenAI