You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Wrappers in Abmarl make a deepcopy of the agents. The original agents in the simulation have their original spaces, and the new, copied agents have the new spaces. In the case of the flatten wrapper, these new spaces are Boxes. The flatten wrapper converts Discrete to Box as a one-hot encoding. Suppose the original space is Discrete(3), then:
0 maps to [1, 0, 0]
1 maps to [0, 1, 0]
2 maps to [0, 0, 1]
When we sample the action space for random actions, it samples the Box, which can produce any of the eight combination of 0s and 1s in a three-element array, namely:
Only three of these eight that I’ve starred are useable in the strict sense of the mapping. The unflatten function for a Discrete space uses np.nonzero(x)[0][0], and here’s at table of what the above arrays map to:
Obviously, [0, 0, 0] will fail because there is no nonzero.
Importantly, only one eighth of the random samples will map to 2. One fourth will map to 1, and one half will map to 0. This has some important implications on exploration, especially if action 2 is the “correct action” throughout much of the simulation.
Solution
This is unique to Discrete spaces. Instead of mapping to a one-hot encoding we could just map to a box of a single element with the appropriate range. Discrete(n) maps to Box(0, n-1, (1,), int) instead of Box(0, 1, (n,), int).
The text was updated successfully, but these errors were encountered:
Wrappers in Abmarl make a deepcopy of the agents. The original agents in the simulation have their original spaces, and the new, copied agents have the new spaces. In the case of the flatten wrapper, these new spaces are Boxes. The flatten wrapper converts Discrete to Box as a one-hot encoding. Suppose the original space is Discrete(3), then:
When we sample the action space for random actions, it samples the Box, which can produce any of the eight combination of 0s and 1s in a three-element array, namely:
Only three of these eight that I’ve starred are useable in the strict sense of the mapping. The unflatten function for a Discrete space uses
np.nonzero(x)[0][0]
, and here’s at table of what the above arrays map to:Implications
Obviously, [0, 0, 0] will fail because there is no nonzero.
Importantly, only one eighth of the random samples will map to 2. One fourth will map to 1, and one half will map to 0. This has some important implications on exploration, especially if action 2 is the “correct action” throughout much of the simulation.
Solution
This is unique to Discrete spaces. Instead of mapping to a one-hot encoding we could just map to a box of a single element with the appropriate range. Discrete(n) maps to Box(0, n-1, (1,), int) instead of Box(0, 1, (n,), int).
The text was updated successfully, but these errors were encountered: