New feature request: reward-hacking onset monitoring hooks

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request
  • new feature request
  • design request
• I have visited the source website
• I have searched through the issue tracker for duplicates

Feature request

Add optional monitoring hooks to Tianshou's trainers that track signals correlated with reward-hacking onset — KL-vs-reference acceleration, entropy-collapse trend, and advantage-distribution drift (Wasserstein vs. a rolling baseline) — logged alongside existing training metrics. Related code: https://github.com/Aarav500/flight-recorder (Apache-2.0), which implements these extractors already (currently for GRPO/TRL-style training, should generalize).

Motivation

Reward hacking is a common failure mode in RL training, and there's currently no standard Tianshou hook for monitoring it. This comes from work on two reward-hacking-detection benchmarks (RHOB, Flight Recorder). One relevant finding: these signals reliably audit hacking after the fact but don't reliably give early warning at a usable false-positive rate (paper: "Audit or Early Warning? A Benchmark for Online Detectors of Reward Hacking").

Proposing before attempting a PR — happy to share methodology/code if there's interest.