如何使用贝尔曼期望方程计算价值函数 V(s)？

In section 2.3.2, "Markov Reward Process," the book introduces the "value function V(s)," which states: "The value function V(s) is the expected return for the state s." A simple method to estimate V(s) is the Monte Carlo method, which involves sampling a large number of trajectories using the state transition matrix P to approximate the expected return for a given state s. In addition to the Monte Carlo method, there are many other methods for computing V(s), such as the Bellman expectation equation and the inverse matrix method, which will be discussed later.