Incorrectly Normalized Importance Weights in Prioritized Experience Replay

In the original PER paper, the priority importance weights are normalized by the max weight. However, in the code, what is being called "max_weight" is actually the normalized weight minimum...

Was this done intentionally? Wouldn't this make the importance weights much larger than they should be? Not just larger, if the priorities were not already normalized, this would put many of them at a value above 1.0

Additionally, I understand fixing this would require a MaxSegTree. It would replace the MinSegTree.

https://github.com/openai/baselines/blob/master/baselines/deepq/replay_buffer.py#L159

`#

class PrioritizedReplayBuffer(ReplayBuffer): .... .... def sample(self, batch_size, beta): .... assert beta > 0 idxes = self._sample_proportional(batch_size)

    weights = []
    p_min = self._it_min.min() / self._it_sum.sum()
    max_weight = (p_min * len(self._storage)) ** (-beta)

    for idx in idxes:
        p_sample = self._it_sum[idx] / self._it_sum.sum()
        weight = (p_sample * len(self._storage)) ** (-beta)
        weights.append(weight / max_weight)
    weights = np.array(weights)
    encoded_sample = self._encode_sample(idxes)
    return tuple(list(encoded_sample) + [weights, idxes])

`