storage/remote: ensure metadata instrumentation make sense for PRW2

Context

Part of https://github.com/prometheus/prometheus/issues/16944

The four metadata metrics in queue_manager.go were designed in #6815 for the PRW1 metadata_config.send side channel, where metadata is:

• sent in separate requests containing only MetricMetadata entries,
• per metric family (not per series),
• emitted once per send_interval (default 1m).

prometheus_remote_storage_metadata_total
prometheus_remote_storage_metadata_failed_total
prometheus_remote_storage_metadata_retried_total
prometheus_remote_storage_metadata_bytes_total

In PRW2 metadata is a field of every writev2.TimeSeries, always encoded, attached per series, in the same request as samples. The old model does not map onto the new one, and today's code reflects that confusion.

This issue is to decide what these metrics should mean (or whether they should exist) for PRW2 before we call PRW2 stable — metric names and semantics are user-facing API, so changing them after GA is a breaking change.

1. Three of the four metrics are permanently 0 under PRW2

A flat-zero counter is worse than an absent one: it reads as "metadata is perfectly healthy".

Note metadata_failed_total is arguably not computable in PRW2: the Written-* response headers only report samples, histograms and exemplars, so there is no receiver-reported metadata stat to diff against, the way updateMetrics does for the other types.

2. metadata_total silently changes meaning by ~3 orders of magnitude

• v1 rate ≈ metric_families / send_interval (e.g. ~500/60s).
• v2 rate ≈ samples rate, because every batched series that carries metadata is counted, on every send attempt.

Same metric name, same queue, wildly different magnitude. Any dashboard/alert/report comparing metadata_total to samples_total, or trending it across a v1 → v2 migration, breaks silently.

3. bytes_total HELP text is wrong under PRW2

"The total number of bytes of data (not metadata) sent by the queue after compression."

Under PRW2, help/unit strings live in the symbols table of the very same request, so metadata bytes are counted in bytes_total — while metadata_bytes_total stays 0.

4. The counting rule in populateV2TimeSeries is incoherent

In populateV2TimeSeries:

• type-and-unit-labels branch: type and unit are always written into the request, but nPendingMetadata++ only happens if help exists → undercount.
• default branch: a Metadata{Type: UNKNOWN} message is still encoded and sent → counted as nothing.
• So the counter is neither "series carrying metadata" nor "series with complete metadata".

Also, nUnexpectedMetadata (a tMetadata record reaching the v2 path, which should be impossible) is only logged as a warning — no counter, so it is invisible in monitoring.

5. Out-of-the-box PRW2 reports zero metadata while logging the opposite

With defaults (no metadata-wal-records, no type-and-unit-labels), seriesMetadata is never populated, so metadata_total == 0 forever — yet we log:

usage of 'metadata_config.send' is redundant when using remote write v2 (or higher) as metadata will always be gathered from the WAL and included for every series within each write request

which is only true with the opt-in metadata-wal-records feature flag. A user migrating v1 → v2 sees metadata_total go to zero, reads the log saying it's fine, and has no signal telling them which is true.

6. The question users actually need answered is coverage, and we don't expose it

For PRW2 the interesting questions are:

1. Is metadata flowing at all, and where does it come from (WAL records / type-and-unit labels / nothing)?
2. What fraction of sent series carry type / unit / help?

The receiver side already has prometheus_api_remote_write_without_metadata_appended_samples_total. There is no sender-side counterpart.