Prometheus-operator not supported other slack compatibility config (Mattermost)
Author: antonshumeyko-seCreated Sep 10, 2026Updated Sep 14, 2026
Labelskind/support
Is there an existing issue for this?
- I have searched the existing issues
What happened?
Description
Not work update_message for mattermost
Steps to Reproduce
slack_configs:
- channel: '#alerts-staging'
send_resolved: true
title: '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] Monitoring Event Notification'
api_url: https://mymat.mycompany.us/api/chat.postMessage
#app_url: https://mymat.mycompany.us/api/chat.postMessage
http_config:
authorization:
credentials: "mypasswd"
update_message: trueI found hard code https://github.com/prometheus-operator/prometheus-operator/blob/5c7146f738f24175a739a59479a0f8d27b671a36/pkg/alertmanager/amcfg.go#L2919 https://github.com/search?q=repo%3Aprometheus%2Falertmanager+https%3A%2F%2Fslack.com%2Fapi%2Fchat.postMessage&type=code
Expected Result
All work config
Actual Result
ts=2026-09-10T08:25:44.240270083Z level=info caller=/workspace/pkg/alertmanager/operator.go:617 msg="sync alertmanager" component=alertmanager-controller key=monitoring/kube-prometheus-stack-alertmanager
ts=2026-09-10T08:25:44.248611639Z level=error caller=/workspace/pkg/operator/resource_reconciler.go:691 msg="Unhandled Error" logger=UnhandledError err="sync \"monitoring/kube-prometheus-stack-alertmanager\" failed: provision alertmanager configuration: failed to initialize from secret: update_message can only be used with bot tokens. api_url must be set to https://slack.com/api/chat.postMessage"
ts=2026-09-10T08:27:06.171022755Z level=info caller=/workspace/pkg/alertmanager/operator.go:617 msg="sync alertmanager" component=alertmanager-controller key=monitoring/kube-prometheus-stack-alertmanager
Prometheus Operator Version
Name: kube-prometheus-stack-operator
Namespace: monitoring
CreationTimestamp: Fri, 27 Dec 2024 16:06:47 +0500
Labels: app=kube-prometheus-stack-operator
app.kubernetes.io/component=prometheus-operator
app.kubernetes.io/instance=kube-prometheus-stack
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=kube-prometheus-stack-prometheus-operator
app.kubernetes.io/part-of=kube-prometheus-stack
app.kubernetes.io/version=88.4.0
chart=kube-prometheus-stack-88.4.0
helm.toolkit.fluxcd.io/name=kube-prometheus-stack
helm.toolkit.fluxcd.io/namespace=flux-system
heritage=Helm
release=kube-prometheus-stack
Annotations: deployment.kubernetes.io/revision: 6
meta.helm.sh/release-name: kube-prometheus-stack
meta.helm.sh/release-namespace: monitoring
Selector: app=kube-prometheus-stack-operator,release=kube-prometheus-stack
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=kube-prometheus-stack-operator
app.kubernetes.io/component=prometheus-operator
app.kubernetes.io/instance=kube-prometheus-stack
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=kube-prometheus-stack-prometheus-operator
app.kubernetes.io/part-of=kube-prometheus-stack
app.kubernetes.io/version=88.4.0
chart=kube-prometheus-stack-88.4.0
heritage=Helm
release=kube-prometheus-stack
Service Account: kube-prometheus-stack-operator
Containers:
kube-prometheus-stack:
Image: quay.io/prometheus-operator/prometheus-operator:v0.93.1
Port: 10250/TCP (https)
Host Port: 0/TCP (https)
Args:
--kubelet-service=kube-system/kube-prometheus-stack-kubelet
--kubelet-endpoints=true
--kubelet-endpointslice=false
--localhost=127.0.0.1
--prometheus-config-reloader=quay.io/prometheus-operator/prometheus-config-reloader:v0.93.1
--config-reloader-cpu-request=0
--config-reloader-cpu-limit=0
--config-reloader-memory-request=0
--config-reloader-memory-limit=0
--thanos-default-base-image=quay.io/thanos/thanos:v0.42.4
--secret-field-selector=type!=kubernetes.io/dockercfg,type!=kubernetes.io/service-account-token,type!=helm.sh/release.v1
--web.enable-tls=true
--web.cert-file=/cert/cert
--web.key-file=/cert/key
--web.listen-address=:10250
--web.tls-min-version=VersionTLS13
Liveness: http-get https://:https/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get https://:https/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
Environment:
GOGC: 30
Mounts:
/cert from tls-secret (ro)
Volumes:
tls-secret:
Type: Secret (a volume populated by a Secret)
SecretName: kube-prometheus-stack-admission
Optional: false
Node-Selectors: <none>
Tolerations: <none>
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available True MinimumReplicasAvailable
OldReplicaSets: kube-prometheus-stack-operator-7f6dcf9fbd (0/0 replicas created), kube-prometheus-stack-operator-845487474d (0/0 replicas created), kube-prometheus-stack-operator-7d6797d48d (0/0 replicas created), kube-prometheus-stack-operator-856dc8cbf5 (0/0 replicas created), kube-prometheus-stack-operator-d6497b5b8 (0/0 replicas created)
NewReplicaSet: kube-prometheus-stack-operator-65774d6464 (1/1 replicas created)
Events: <none>Kubernetes Version
clientVersion:
buildDate: "2026-03-18T19:18:44Z"
compiler: gc
gitCommit: 8b2bf66ce7018f6e13e8767624d4ce7768a8a2e5
gitTreeState: clean
gitVersion: v1.34.6
goVersion: go1.24.13
major: "1"
minor: "34"
platform: linux/amd64
kustomizeVersion: v5.7.1
serverVersion:
buildDate: "2025-11-11T19:00:39Z"
compiler: gc
emulationMajor: "1"
emulationMinor: "34"
gitCommit: 8cc511e399b929453cd98ae65b419c3cc227ec79
gitTreeState: clean
gitVersion: v1.34.2
goVersion: go1.24.9
major: "1"
minCompatibilityMajor: "1"
minCompatibilityMinor: "33"
minor: "34"
platform: linux/amd64Kubernetes Cluster Type
kubeadm
How did you deploy Prometheus-Operator?
helm chart:prometheus-community/kube-prometheus-stack
Manifests
prometheus-operator log output
ts=2026-09-10T08:25:44.240270083Z level=info caller=/workspace/pkg/alertmanager/operator.go:617 msg="sync alertmanager" component=alertmanager-controller key=monitoring/kube-prometheus-stack-alertmanager
ts=2026-09-10T08:25:44.248611639Z level=error caller=/workspace/pkg/operator/resource_reconciler.go:691 msg="Unhandled Error" logger=UnhandledError err="sync \"monitoring/kube-prometheus-stack-alertmanager\" failed: provision alertmanager configuration: failed to initialize from secret: update_message can only be used with bot tokens. api_url must be set to https://slack.com/api/chat.postMessage"
ts=2026-09-10T08:27:06.171022755Z level=info caller=/workspace/pkg/alertmanager/operator.go:617 msg="sync alertmanager" component=alertmanager-controller key=monitoring/kube-prometheus-stack-alertmanagerAnything else?
No response
Source: prometheus-operator/prometheus-operator