Dashboard forces prometheusEndpointSSLVerify: false — no way to give the mgr a custom CA

Is this a bug report or feature request?

• Feature Request

What should the feature do:

Honor a custom CA bundle for the ceph-mgr pod, the same way CephObjectStore.spec.gateway.caBundleRef already does for the RGW — e.g. CephCluster.spec.mgr.caBundleRef, or a cluster-wide CephCluster.spec.caBundleRef that applies to all daemon pods.

The machinery already exists in the operator, it is just not wired into the mgr pod spec. From pkg/operator/ceph/object/spec.go:

// createCaBundleUpdateInitContainer()
return v1.Container{
    Name:    "update-ca-bundle-initcontainer",
    Command: []string{"/bin/bash", "-c"},
    Args: []string{
        fmt.Sprintf("/usr/bin/update-ca-trust extract; cp -rf %s/* %s",
            caBundleExtractedDir, updatedCaBundleDir),
    },
    ...
}

plus the two volumes added in makeRGWPodSpec() and the CURL_CA_BUNDLE environment variable set on the daemon container.

What is use case behind this feature:

The Ceph dashboard runs inside ceph-mgr and makes outbound HTTPS calls — most importantly to the Prometheus API configured through CephCluster.spec.dashboard.prometheusEndpoint. When that endpoint serves a certificate from an internal PKI, there is currently no way to make the mgr trust it, so spec.dashboard.prometheusEndpointSSLVerify has to be set to false. That is the only unverified TLS hop left in our cluster, and it is not a configuration choice — there is no alternative.

Why the obvious alternatives do not work (all measured on a running cluster, Rook v1.20.3 / Ceph v20.2.2):

1. Ceph's own custom-CA path is gated behind cephadm. src/pybind/mgr/dashboard/controllers/prometheus.py:

   复制

   verify = ca_cert_file.name if ca_cert_file else Settings.PROMETHEUS_API_SSL_VERIFY
   ...
   is_cephadm = mgr.get_module_option_ex('orchestrator', 'orchestrator') == 'cephadm'
   if not is_cephadm:
       return Credentials(user, password, ca_cert_file, cert_file, pkey_file)  # all None

   With Rook as the orchestrator ca_cert_file is always None.

2. The setting cannot carry a path instead of a bool: PROMETHEUS_API_SSL_VERIFY = Setting(True, [bool]).

3. The CephCluster CRD offers no way to mount a file or set an env var on the mgr. In the installed CRD: 0 occurrences of volumeMounts, 0 of "env"; spec.mgr only has allowMultiplePerNode, count, hostNetwork, modules, placement and resources.

4. verify=True therefore falls back to the container's system trust store: requests.certs.where() -> /etc/pki/tls/certs/ca-bundle.crt (146 public roots), and /etc/pki/ca-trust/source/anchors/ is empty.

Why this should be a small change: the dashboard uses python-requests, which honors CURL_CA_BUNDLE exactly when verify is True. Measured inside the mgr container (requests 2.25.1, requests/sessions.py):

if verify is True or verify is None:
    verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
              os.environ.get('CURL_CA_BUNDLE'))

So the very environment variable Rook already sets for the RGW would make prometheusEndpointSSLVerify: true work as-is, with no change needed on the Ceph side.

Related issues, both closed as wontfix, both describing this same gap from the user side: #15336 ("certificate verify failed: unable to get local issuer certificate" in the mgr) and #9584 (asks for either a verify toggle or "enabling Rook pods to use the host operating system's CA trust bundle"). #14930 is relevant for the implementation, since the init container assumes update-ca-trust is present.

Environment:

• OS: Debian GNU/Linux 13 (trixie)
• Kernel: 6.12.95+deb13-arm64
• Cloud provider or hardware configuration: single arm64 node (VPS)
• Rook version: v1.20.3 (docker.io/rook/ceph:v1.20.3)
• Storage backend version: Ceph v20.2.2 (quay.io/ceph/ceph:v20.2.2)
• Kubernetes version: v1.36.2+k3s1
• Kubernetes cluster type: k3s
• Storage backend status: HEALTH_WARN 8 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 (expected on a single-node cluster, unrelated to this request)