Should the reported metric mean disclose how many samples it was computed over?

A semantics question about the reported metric mean, found while auditing eval frameworks' measurement contracts.

The observation

With raise_exceptions=False (the default), a failed metric call returns np.nan (src/ragas/executor.py ~L84: return counter, np.nan), and the aggregated result is safe_nanmean over per-sample values (src/ragas/dataset_schema.py ~L445). EvaluationResult.__repr__ prints the mean to 4 decimals — but I don't see the success count or failure count anywhere in the default output.

The question

If 100 samples are evaluated and 90 metric calls fail on API errors, print(result) still shows a perfectly plausible faithfulness score — the mean over the 10 surviving samples. Is that the intended contract ("mean over available samples"), and if so, would it make sense to expose n_success / n_total (in the repr, a warning threshold, or a result field)?

The scenario I care about: comparing two prompt versions where the failure sets differ (rate limits, timeouts). Today's output cannot distinguish "the model got better" from "a different subset survived." A denominator field would make version comparisons interpretable without re-running with raise_exceptions=True.

Happy to send a minimal repro (a metric that raises for a chosen subset of rows) or a small PR adding the counts to the repr/result. Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of eval/telemetry semantics — this is the finding from the Ragas row worth asking about directly. Not a bug claim; a contract question.