Exercise4.9

[soonjune]

I think there is something wrong with the implementation. The optimal policy should be 1 for all states when p_h = 0.55. There is a big bet around 80 and I cannot find any reason for this behavior.

[ufownl]

I got the same result as @soonjune .

[GeorgGroenendaal]

Does anybody know why this happens, I get similar results with this implementation and my own. Is this because it does not result in a stable solution?

[StupidI]

Does anybody know why this happens, I get similar results with this implementation and my own. Is this because it does not result in a stable solution?

Due to the limited accuracy, it is not reliable to compare equality, some value is actually the same. So we should limit the precision when we compare the value of q(s,a)——> just change the part of "np.argmax"