ULTRA Add scenario density information in GB (#730)

* Update evaluate and train to eval per episode * Fix tests and change normalization * Fix formatting * Record episode for train/eval at the same time * Train performance averages last eval_rate episodes * Add gap_mode and calculate generalization gap * Enable explore during eval for better comparison * Fix formatting * Adjust ci (#649) (#650) removes eager update strategy in ci automatically upgrades numpy version in ci * Fix averaging in record_episode * Only average scalars in record_episode * Check for scalar when calculating gap * Fix formatting * Fix bugs introduced by merge * Fix calculate_gap * Merge 0.4.15 version changes to master * Fix averaging over episodes * Fix formatting * Check eval_episodes is 0 before evaluation * Make evaluations run in parallel to training * Evaluate on train and test without exploration * Make train/test occur in one ray.get * Empty commit: create ultra-develop * Add on_shoulder event (#714) (#717) * add on_shoulder in events in observations * update changelogs * display grade intervals (episodes) * Update open agent example * Update changelog * Format * Remove episode_count * Remove episode_count from tests * Showbase refactor (#706) * First pass at ShowBase refactor into Renderer; Also added Provider base class. WIP * Updated the pybullet examples. +comments. * Added changelog entry. * find collidee by vehicle.id, not node in scene graph. misc. minor cleanup. * oops, had wrong issue number in CHANGELOG. * Made ShowBaseInstance class private. * Fixups from review: actually use Provider! * Added some comments to explain why overlapping SMARTS is still not a good idea. * fixups from review (better encapsulation) * fixed oops * don't use ShowBase clock for main simulation clock. * hide other sim_root nodes when rendering * Added simple multiple-renderers test. * WIP: Scenario density information available during training * isolate density extraction from non gb runs * WIP: Evaluate updated to record test densities * make done_creteria independent from events (#728) * make done_creteria independent from events * update changelog * map offset fixes (#727) * Use possibly-shifted road network file when starting Sumo. * use unique name for shifted net file just in case. * Don't use a unique shifted_net filename every time afterall to avoid accumlation of cruft. * updated CHANGELOG * Fixed Changelog location. * Add CHANGELOG line for previous map offset fix saying that scenario maps need to be rebuilt. * Describe map creation (#609 #689) (#701) * Added description for map creation * Mentioned paper in docs * change mention to paper to front page * update changelog * added description of lane connection * minor typos * add sub-headings Co-authored-by: Stephen Hockema <[email protected]> * Make envision server can be killed by terminal signal * Remove averaging over training episodes * Restructure evaluation_check with eval_mode = False * Fix formatting * Fix headless argument in ULTRA scripts * Refactored train, evaluate, coordinator & scenario data available from evaluate * updated config file * try split ci test method * updated gb tests & split evaluate tests into single and multi * clean up * formatted * fix tests * Add rollout_fragment_length and max_episode_steps to rllib * Add new flags to documentation * fix ci * Fix flag in example * Renderer Optional (#729) * First pass adding renderer_optional param to SMARTS. * updated CHANGELOG * recategorized the renderer changes in CHANGELOG * Fixed location of changelog entry. * ok, I'm going to be more careful with CHANGELOG changes going forward! * Added new RendererException (per review) * Added shifted_map-AUTOGEN.net.xml to .gitignore (due to already-merged "maps_offset_fixes" PR). * Only create Renderer on demand (if camera-based sensor added). * Ensure evaluation is not run in test_train.py * Check if log_dir doesn't exist in test_train.py * Fix typo in argument and set eval_episodes to 0 * Check if not exists for paths * Fix formatting * Update evaluate and train to evaluate per episode (#639) * Update evaluate and train to eval per episode * Fix tests and change normalization * Fix formatting * Record episode for train/eval at the same time * Train performance averages last eval_rate episodes * Add gap_mode and calculate generalization gap * Enable explore during eval for better comparison * Fix formatting * Fix averaging in record_episode * Only average scalars in record_episode * Check for scalar when calculating gap * Fix formatting * Fix bugs introduced by merge * Fix calculate_gap * Fix averaging over episodes * Fix formatting * Check eval_episodes is 0 before evaluation * Evaluate on train and test without exploration * Make train/test occur in one ray.get * Remove episode_count * Remove episode_count from tests * Remove averaging over training episodes * Restructure evaluation_check with eval_mode = False * Fix formatting * try split ci test method * Check for valid eval_episodes inside evaluation_check * Removed averaging feature of record_episode * Fix formatting Co-authored-by: Jenish Patel <[email protected]> * Merge with ultra-rllib-speedup-tests * Rename evaluation_task_id for training * fix ci * split evaluate tests into heavy and light * Update smarts/core/sumo_road_network.py * Simplify argument to ray.wait(...) * Remove print statements * Fix defaults * Fix eval-rate in docs * Scenario density analysis now available in non-gb mode * static evaluation models for test * refactored train/evaluate/episodes to store scenarios inside of episode.info object * fixed gb test error Co-authored-by: AlexLewandowski <[email protected]> Co-authored-by: Jeffer Jingfei Peng <[email protected]> Co-authored-by: Tucker <[email protected]> Co-authored-by: christianjans <[email protected]> Co-authored-by: sah-huawei <[email protected]> Co-authored-by: Stephen Hockema <[email protected]> Co-authored-by: Valax Kong <[email protected]> Co-authored-by: Tucker Alban <[email protected]>
huawei-noah · Apr 12, 2021 · 18663af · 18663af
1 parent e73ac87
commit 18663af
Show file tree

Hide file tree

Showing 61 changed files with 1,658 additions and 930 deletions.
diff --git a/.github/workflows/ci-ultra-tests.yml b/.github/workflows/ci-ultra-tests.yml
@@ -41,7 +41,8 @@ jobs:
           --ignore=./tests/test_episodes.py \
           --ignore=./tests/test_scenarios.py \
           --ignore=./tests/test_social_vehicles.py \
-          --ignore=./tests/test_gb_train.py
+          --ignore=./tests/test_rllib_train.py \
+          --ignore=./tests/test_gb_train.py \
 
   test-light-base-tests:
     runs-on: ubuntu-18.04
@@ -77,7 +78,6 @@ jobs:
           ./tests/ \
           --ignore=./tests/test_ultra_package.py \
           --ignore=./tests/test_train.py \
-          --ignore=./tests/test_rllib_train.py \
           --ignore=./tests/test_evaluate.py \
           --ignore=./tests/test_analysis.py \
 

diff --git a/.gitignore b/.gitignore
@@ -97,6 +97,7 @@ venv
 *.rou.alt.xml
 *.rou.xml
 *.trips.xml
+shifted_map-AUTOGEN.net.xml
 *.egg
 *.glb
 *flamegraph-perf.log

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -12,15 +12,27 @@ Copy and pasting the git commit messages is __NOT__ enough.
 ### Added 
 - Added `sanity-test` script and asked new users to run `sanity-test` instead of `make test` to ease the setup
 process
+- Added `on_shoulder` as part of events in observation returned from each step of simulation
+- Added description of map creation and how to modify the map to allow users to create their own traffic routes in docs
+- Added reference to SMARTS paper in front page of docs
+- Only create `Renderer` on demand if vehicles are using camera-based sensors. See issue #725.
+### Changed
+- Refactored SMARTS class to not inherit from Panda3D's ShowBase; it's aggregated instead. See issue #597.
+### Fixed
+- Fixed the bug of events such as off_road not registering in observation when off_road is set to false in DoneCriteria
+- Fixed sumo road network offset bug for shifted maps.  See issue #716.
+
+### Fixed
+- Update `ego_open_agent` to use the package instead of the zoo directory version.
 
 ## [0.4.15] - 2021-03-18
 ### Added
 - This CHANGELOG as a change log to help keep track of changes in the SMARTS project that can get easily lost.
 - Hosted Documentation on `readthedocs` and pointed to the smarts paper and useful parts of the documentation in the README.
-- Running imitation learning will now create a cached history_mission.pkl file in scenario folder that stores 
+- Running imitation learning will now create a cached `history_mission.pkl` file in scenario folder that stores 
 the missions for all agents.
 - Added ijson as a dependency. 
-- Added cached_property as a dependency.
+- Added `cached_property` as a dependency.
 ### Changed
 - Lowered CPU cost of waypoint generation. This will result in a small increase in memory usage.
 - Set the number of processes used in `make test` to ignore 2 CPUs if possible.
@@ -33,6 +45,7 @@ we now initially shift road networks (maps) that are offset back to the origin
 using [netconvert](https://sumo.dlr.de/docs/netconvert.html).
 We adapt Sumo vehicle positions to take this into account to allow Sumo to continue
 using the original coordinate system.  See Issue #325.
+    - This fix will require all Scenarios to be rebuilt (`scl scenario build-all --clean ./scenarios`).
 - Cleanly close down the traffic history provider thread. See PR #665.
 - Improved the disposal of a SMARTS instance. See issue #378.
 - Envision now resumes from current frame after un-pausing.

diff --git a/docs/_static/chain_two_way.png b/docs/_static/chain_two_way.png
diff --git a/docs/_static/connection_icon.png b/docs/_static/connection_icon.png
diff --git a/docs/_static/create_connection.png b/docs/_static/create_connection.png
diff --git a/docs/_static/inspect.png b/docs/_static/inspect.png
diff --git a/docs/_static/map_lane.png b/docs/_static/map_lane.png
diff --git a/docs/index.rst b/docs/index.rst
@@ -3,6 +3,8 @@ Welcome to SMARTS!
 
 Scalable Multi-Agent RL Training School (**SMARTS**) is an autonomous driving platform for reinforcement learning reasearch.
 
+Check out the paper at `Link SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving <https://arxiv.org/abs/2010.09776>`_ for background on some of the project goals.
+
 .. image:: _static/smarts_envision.gif
 
 .. literalinclude:: minimal.py

diff --git a/docs/sim/scenario_studio.rst b/docs/sim/scenario_studio.rst
@@ -46,7 +46,7 @@ We just run a `scenario.py` file as a regular Python script to generate scenario
   python3 scenario/scenario.py
 
 IMPORTANT: if you want to train model on one scenario, remember to set the `end` time of flow larger or equal to your expected
-training time, since the smarts will continue the flow after each `reset` called. However, if there are multi scenarios to train
+training time, since SMARTS will continue the flow after each `reset` call. However, if there are multiple scenarios to train
 for one worker, you can relax this restriction since after the scenario change, the flow will also be reset to the beginning time.
 
 =================
@@ -83,7 +83,7 @@ The Scenario Studio of SMARTS also allows generation of *friction map* which con
   )
 
 ===============
-Create new maps
+Edit SUMO maps
 ===============
 
 To enrich your training datasets, you can edit your own map through [SUMO's NETEDIT](https://sumo.dlr.de/docs/NETEDIT.html) and export it in a map.net.xml format.
@@ -93,3 +93,65 @@ And if you have an additional file you wish to turn into a map you can use the c
 
   python3 -m smarts.sstudio.sumo2mesh dataset_public/2lane_sharp/map.net.xml dataset_public/2lane_sharp/map.glb --format=glb
   python3 -m smarts.sstudio.sumo2mesh dataset_public/2lane_sharp/map.net.xml dataset_public/2lane_sharp/map.egg --format=egg
+
+
+Create map
+---------------
+
+First, to start `netedit`, run the following on terminal:
+
+.. code-block:: bash
+
+  netedit
+
+On the top left bar, "file" -> "new network" to create a new map. 
+
+Use shortcut key "e" to change to edge mode. Click "chain" and "two-way" icons located on the far right of top tool bar, shown in the image below:
+
+.. image:: ../_static/chain_two_way.png
+
+Then click on map to start creating new edges and lanes.
+
+Edit existing map
+----------------------
+
+"file" -> "Open Network..." to open an existing map. 
+
+Click on the inspect icon to enable inspect mode
+
+.. image:: ../_static/inspect.png
+
+Click on any edge to inspect detail and modify properties.
+
+.. image:: ../_static/map_lane.png
+
+The selected block is an edge with id "gneE72". It contains 3 lanes with lane index 0, 1, 2.
+
+To modify the properties, for example change the number of lanes to 2 lanes by changing 3 to 2 on the "numLanes" field, and press
+"enter" to make the change. Then press "ctrl+s" to save. Finally, make sure to rebuild the scenario.
+
+.. code-block:: bash
+
+  scl scenario build --clean <path-to-scenario-folder>
+
+To create custom connections between edges, first click the following icon on top bar:
+
+.. image:: ../_static/connection_icon.png
+
+The first lane you select would be the source lane, highlighted in blue. Then select other lanes as target lanes to connect to.
+
+.. image:: ../_static/create_connection.png
+
+
+=====================
+Create traffic routes
+=====================
+
+For example, using the following `Route` definition:
+
+.. code-block:: python
+  
+  Route(begin=("gneE72", 0, "random"), end=("edge2", 1, "max"),)
+
+`begin=("gneE72", 0, "random")` defines the route to start on edge with id `gneE72` and at lane index `0`, 
+which is the same lane as the selected lane in the figure above. `"random"` here specifies the amount of offset on the lane to start the route.
diff --git a/envision/server.py b/envision/server.py
@@ -428,6 +428,9 @@ def run(scenario_dirs, max_capacity_mb=500, port=8081):
     signal.signal(
         signal.SIGINT, lambda signal, _: ioloop.add_callback_from_signal(on_shutdown)
     )
+    signal.signal(
+        signal.SIGTERM, lambda signal, _: ioloop.add_callback_from_signal(on_shutdown)
+    )
     ioloop.start()
 
 

diff --git a/examples/ego_open_agent.py b/examples/ego_open_agent.py
@@ -12,7 +12,12 @@
 
 from smarts.core.utils.episodes import episodes
 
-open_agent = importlib.import_module("zoo.policies.open-agent.open_agent")
+try:
+    open_agent = importlib.import_module("open_agent")
+except ModuleNotFoundError as e:
+    raise ModuleNotFoundError(
+        f"Ensure that the open-agent has been installed with `pip install open-agent"
+    )
 
 from examples import default_argument_parser
 

diff --git a/smarts/core/chassis.py b/smarts/core/chassis.py
@@ -670,7 +670,9 @@ def _set_road_friction(self, current_simulation_time):
         exists and the vehicle is located in the defined regions
         in scenario file.
         """
-        pos = Point(self.pose.as_panda3d()[0])
+        pos = Point(
+            self.pose.as_panda3d()[0]
+        )  # TODO: does this need to be "as_panda3d"?
         # A check to see if we are in a surface patch.
         for surface_patch in self._friction_map:
             if pos.within(surface_patch["zone"].to_geometry()) and (

diff --git a/smarts/core/events.py b/smarts/core/events.py
@@ -22,9 +22,10 @@
 
 class Events(NamedTuple):
     collisions: bool
-    off_route: bool
-    reached_goal: bool
-    reached_max_episode_steps: bool
     off_road: bool
+    off_route: bool
+    on_shoulder: bool
     wrong_way: bool
     not_moving: bool
+    reached_goal: bool
+    reached_max_episode_steps: bool
diff --git a/smarts/core/motion_planner_provider.py b/smarts/core/motion_planner_provider.py
@@ -24,11 +24,11 @@
 from .bezier_motion_planner import BezierMotionPlanner
 from .controllers import ActionSpaceType
 from .coordinates import Heading, Pose
-from .provider import ProviderState
+from .provider import Provider, ProviderState
 from .vehicle import VEHICLE_CONFIGS, VehicleState
 
 
-class MotionPlannerProvider:
+class MotionPlannerProvider(Provider):
     def __init__(self):
         self._is_setup = False
 

diff --git a/smarts/core/provider.py b/smarts/core/provider.py
@@ -18,12 +18,14 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 from dataclasses import dataclass, field
-from typing import List
+from typing import List, Set
 
 import numpy as np
 
 from .coordinates import BoundingBox, Pose
 from .vehicle import VehicleState
+from .scenario import Scenario
+from .controllers import ActionSpaceType
 
 
 @dataclass
@@ -46,3 +48,29 @@ def filter(self, vehicle_ids):
                 del self.vehicles[index]
             except ValueError:
                 continue
+
+
+class Provider:
+    """A Provider tracks a (sub)set of vehicles that all share the same action space.
+    This is a base class (interface) from which all Providers should inherit."""
+
+    ## TAI: Consider renaming to VehicleSet or somesuch.
+
+    @property
+    def action_spaces(self) -> Set[ActionSpaceType]:
+        raise NotImplementedError
+
+    def setup(self, scenario: Scenario) -> ProviderState:
+        raise NotImplementedError
+
+    def step(self, actions, dt, elapsed_sim_time) -> ProviderState:
+        raise NotImplementedError
+
+    def sync(self, provider_state: ProviderState):
+        raise NotImplementedError
+
+    def reset(self):
+        raise NotImplementedError
+
+    def teardown(self):
+        raise NotImplementedError