[Bug]: Improper Handling of Invalid abci_query Paths Leading to Telemetry and Metrics Disruption

Is there an existing issue for this?

• I have searched the existing issues

What happened?

This bug in the Cosmos SDK telemetry system occurs when an invalid or unknown query path is passed to the abci_query endpoint. This causes the node to dynamically register invalid metrics with Prometheus, leading to disruptions in the /metrics endpoint and making it unusable until the node is restarted. This issue impacts anyone using Prometheus or other monitoring tools for observability of a Cosmos SDK node, as it prevents proper telemetry collection and operational visibility.

Cosmos SDK Version

v0.50.11

How to reproduce?

1. Start a Cosmos SDK Node: Ensure the Cosmos SDK node is running with telemetry and metrics enabled.

2. Send an Invalid abci_query Request: Execute the following command to send an invalid abci_query request with a malformed path (replace with a placeholder string):

curl -X GET "http://localhost:5702/abci_query?path=\"/cosmos/slashing/v1beta1/signing_infos/<validator-consensus-address>\""

1. Scrape Metrics from /metrics Endpoint: Attempt to retrieve metrics from the /metrics endpoint:

curl http://localhost:6702/metrics

1. Observe the Error: The /metrics endpoint will return an error and stop reporting valid metrics:

An error has occurred while serving metrics:
2 error(s) occurred:
* "cosmos__cosmos_slashing_v1beta1_signing_infos_<validator_consensus_address>" is not a valid metric name
* "cosmos_query__cosmos_slashing_v1beta1_signing_infos_<validator_consensus_address