Best way to reassign missions to different agents?

[ajlangley]

High Level Description
I created a four-way intersection scenario and made a gym environment using HiWayEnv. Given n agents an m missions, I'm trying to find the best way to randomly assign different missions to different agents each episode.

Desired SMARTS version
1.0.3

Operating System
Ubuntu 20.04

Problems
The first issue is that it looks like HiWayEnv expects one of:

1. No mission and >0 agents, so that missions are generated and assigned randomly,
2. >0 missions and 1 agent, where missions are again assigned randomly,
3. m == n, where I'm assuming missions are assigned in order (ith agent interface is assigned the ith mission).

The case of m != n seems a little more complicated, so let's assume m == n. The ways I've thought of shuffling the missions are

1. Shuffling env.scenario._missions manually on reset() (don't know if this works),
2. Creating many scenario directories with different permutations of missions,
3. Using the SMARTS simulator directly and creating a new Scenario object on reset with the desired mission assignment.

I feel like 1 is hacky, 2 is messy, and 3 is most likely inefficient. Is there a preferred way to do this?

[Gamenot]

Hello @ajlangley, there is not a current standard way of managing permutation. This is an issue we have been deferring.

Current issue

I assume that the Scenario.scenario_variations iterator (used by HiWayEnv.reset) is not working as you expect? Specifically, the missions returned do not contain permutations of mission order?

1. No mission and >0 agents, so that missions are generated and assigned randomly,
2. >0 missions and 1 agent, where missions are again assigned randomly,
3. m == n, where I'm assuming missions are assigned in order (ith agent interface is assigned the ith mission).

The case of m != n seems a little more complicated, so let's assume m == n. The ways I've thought of shuffling the missions are

With respect to your considerations:

1. This is not currently possible, Scenario should only represent what is needed to create the initial state of the scenario within the simulation on reset. I can see that it would be best for us to rename SMARTS.scenario to SMARTS.scenario_spec to avoid confusion.
2. (your 2) Explicitly generating permutations of scenarios to different directories. This is confirmed to work but ends up with an inefficient use of disk space. Because we have not yet set up resource references it requires the map file to also exist in each of those directories.
3. (your 3) Using the SMARTS simulator directly and creating a new Scenario object to pass to reset with the desired mission assignment. As previously mentioned Scenario.scenario_variations is the utility used to determine the missions. Replacing the generation of scenarios would allow to you manually select mission order.

Future approaches

We had also considered future solutions to make the missions less mysterious.

1. Implement named missions that can be requested through the agent interface.
2. Defer mission generation and have users pick their own mission order. (this does not work well with the gym interface)
3. Require permutation of missions to be explicitly declared in scenario.py for each of those cases. (push mission order specification on the user)

[ajlangley]

The way I expected for agent mission reassignment to work would have been that calling an environment's reset method also resulted in a permuted env.scenario._missions dictionary. Indeed, it remained the same after calling reset. That being said, I'm unclear on whether just printing the variable itself would be reflective of whether missions were reassigned. I think the only way to check it for sure would be to check visually in Envision, but I am also having an issue visualizing custom missions on a custom map (which I was planning to make a separate report about), so I can't check that for now. It also doesn't look at first glance like there's any logic in Scenario.scenario_variations that would do this.

As for what would be the best option out of the possible directions for this that you gave, wouldn't 3 require the person designing the scenario to essentially hard code the number of agents that will be in it? This seems undesirable if it is in fact the case. I think 2 may be the best option, and the agent to mission mapping could optionally be passed to the reset method. This would be compliant with the usual gym Env interface because the reset method accepts **kwargs anyway. In this case, the user could either pass the desired mapping, or a random subset of missions could be assigned.

[Gamenot]

To correct myself, I was still thinking in regards to the gym.Env.reset(). The 2nd option could work in terms of gymnasium environments.

• gym<=0.19.0 interface is reset(self)
  • This for gym environments (such as hiway-v0) does not allow us to provide the scenario.
  • We currently cannot upgrade gym
• gym>=0.21.0|gymnasium interface is reset(self, *, seed, options: Dict[str, Any])
  • This interface for the newer gym interface (such as hiway-v1) allows us to provide the reset configuration through options and forward that to SMARTS.reset().

We install gymnasium alongside gym and these are roughly similar in behaviour but with different interface versions:

• gym.make("smarts.env:hiway-v0) or HiWayEnv(...)
• gymnasium.make("smarts.env:hiway-v1", observation_options="unformatted") or HiWayEnvV1(..., observation_options="unformatted")

Since our intention is to move over to using gymnasium environments, it seems viable to provide an override through options.

• options['scenario'] could allow an explicit override of the scenario in hiway-v1.

I think this is possible in the short term to simplify scenario modification.

[ajlangley]

I see. Thanks for the discussion. By override, do you mean that upon calling reset, a new scenario would be generated, with the desired mission mapping, and the scenario that is next in the environment's HiWayEnv._scenarios_iterator would just be ignored?

[ajlangley]

Here's a thought: For reference, I'm looking at line 275 in smarts.core.scenario.py. One way of implementing the above idea would be for the generator to return a function which constructs a new Scenario object with the passed in arguments as defaults to the function. Then in the reset method, all that would need to change is that we'd use a function call when getting the next scenario. Something like

def variations_for_all_scenario_roots(
        scenario_roots, agents_to_be_briefed, shuffle_scenarios=True
    ) -> Generator["Scenario", None, None]:

    # function body

    def make_scenario(missions=concrete_agent_missions):
        return Scenario(
                    scenario_root,
                    traffic_specs=concrete_traffic,
                    missions={
                        **(missions or {}),
                        **concrete_social_agent_missions,
                    },
                    social_agents=concrete_social_agents,
                    surface_patches=surface_patches,
                    traffic_history=concrete_traffic_history,
                )

    return make_scenario

[Gamenot]

I see. Thanks for the discussion. By override, do you mean that upon calling reset, a new scenario would be generated, with the desired mission mapping, and the scenario that is next in the environment's HiWayEnv._scenarios_iterator would just be ignored?

With the current intended change the scenario could be passed on reset in through the options forwarding it to SMARTS.reset. HiWayEnv._scenarios_iterator is ignored in that case. This is useful even if it does not fully solve your request by itself. Working with the entire Scenario results in few issues.

Here's a thought: For reference, I'm looking at line 275 in smarts.core.scenario.py. One way of implementing the above idea would be for the generator to return a function which constructs a new Scenario object with the passed in arguments as defaults to the function. Then in the reset method, all that would need to change is that we'd use a function call when getting the next scenario. Something like

def variations_for_all_scenario_roots(
        scenario_roots, agents_to_be_briefed, shuffle_scenarios=True
    ) -> Generator["Scenario", None, None]:

    # function body

    def make_scenario(missions=concrete_agent_missions):
        return Scenario(
                    scenario_root,
                    traffic_specs=concrete_traffic,
                    missions={
                        **(missions or {}),
                        **concrete_social_agent_missions,
                    },
                    social_agents=concrete_social_agents,
                    surface_patches=surface_patches,
                    traffic_history=concrete_traffic_history,
                )

    return make_scenario

This is mostly reasonable.

The main issue that I see with this approach is that reset is contained. In the case that there are multiple scenarios (e.g. HiWayEnv(scenarios=[loop, 4lane_t])), those scenarios may have completely different mission configurations and the scenario only becomes known after reset (but the missions have to be determined before to be passed into reset.)

It might just be possible if there was something like HiWayEnv.peek_scenario to check information about the next scenario prior to reset. Then it would be possible to provide informed scenario configuration through options.

[ajlangley]

Yeah, the proposed change definitely makes sense. I suppose one could just create a wrapper around the HiWayEnvV1 that shuffles the missions this way.

I agree with the issue you raised about the code snippet I gave. This is definitely better.

[Gamenot]

I have added an issue to rectify issues with the internal permutations generated by the SMARTS scenario.