Improve provenance and default presentation of config composition errors

Problem

Configuration-composition failures should point to the config and Defaults List entry that caused the problem. A Python traceback through Hydra's composition implementation is usually less useful than that provenance. Reporting is currently inconsistent:

• A missing selected config reports the referring config, missing option, and config search path. Merge failures report the config being merged and retain OmegaConf details such as full_key and object_type. These are useful patterns to preserve.
• A mandatory Defaults List choice (group1: ???) reports the group and available options, but not the config containing that entry.
• Conflicting group choices report Multiple values for group1 and override guidance, but not which configs supplied the conflicting choices.
• A non-list defaults value reports the config and expected type only at the end of a long internal traceback in the default CLI output.

These cases were reproduced against current Hydra using --cfg job and existing test configs (unspecified_mandatory_default, error_changing_group, and defaults_not_list).

Scope for Hydra 1.4

Make default composition-error output concise and actionable. Where the information is available, identify the logical config path containing the offending Defaults List entry, the entry/group/option involved, and the relevant conflicting or missing choice. Preserve existing useful context: available options, config search path for lookup failures, and OmegaConf's key/type details for merge failures. Use a sensible fallback when the source cannot be identified (for example, a generated or ConfigStore config); do not invent a file path or line number.

In particular, improve the three gaps above: mandatory ??? choices, conflicting group choices, and malformed defaults values. Audit adjacent composition errors for the same provenance/presentation problem, but keep changes targeted to errors with a concrete missing diagnostic.

Non-goals

• Do not add Python backtraces to ordinary config-composition or command-line override errors.
• Do not change instantiation-error traceback handling; that is #3361.
• Do not change composition semantics or the separate --help behavior for ??? defaults tracked in #2170.
• Do not require physical YAML paths or line numbers when the config source cannot provide them.

Acceptance criteria

• Default CLI output for the examples above names the offending logical config and relevant group/entry, retains useful choices or override guidance, and does not require reading Hydra internals to find the diagnostic.
• Conflicting choices identify their origins when available, rather than only naming the group.
• Existing missing-config and merge-error context remains intact, including search paths, full_key, and object_type where applicable.
• Direct composition API callers still receive meaningful exceptions; HYDRA_FULL_ERROR=1 retains the full native traceback.
• Focused CLI and API regression tests cover the improved cases, including a source without a physical YAML file. Add a user-facing news fragment.

Related

• #3361: instantiation traceback rendering (separate scope).
• #1697: previously improved merge-error config provenance.
• #2170: --help with mandatory ??? defaults (separate behavior).