Parallel observations #1687

Gamenot · 2022-10-28T14:16:44Z

No description provided.

Adaickalavan · 2022-10-31T21:13:18Z

Something to consider in parallel computing: mpi4py, albeit this approach requires significant code changes.

Gamenot · 2022-11-04T19:04:04Z

This changeset has a lot of useful features but the changes are not turning out quite as I hoped. Process communication seems to be slow as determined by the tests.

saulfield · 2022-12-14T22:45:20Z

smarts/core/road_map.py

+        """The default serialization for the road map."""
+        import cloudpickle
+
+        return cloudpickle.dumps(road_map)


How did you get around the issues related to lanepoints that we were seeing with this?

I ended up using a proxy object to format and then reconstruct the road_map.

SMARTS/smarts/core/serialization/default.py

Lines 28 to 80 in edd90db

def dumps(__o):

"""Serializes the given object."""

import cloudpickle

_lazy_init()

r = __o

type_ = type(__o)

# TODO: Add a formatter parameter instead of handling proxies internal to serialization

proxy_func = _proxies.get(type_)

if proxy_func:

r = proxy_func(__o)

return cloudpickle.dumps(r)

def loads(__o):

"""Deserializes the given object."""

import cloudpickle

r = cloudpickle.loads(__o)

if hasattr(r, "deproxy"):

r = r.deproxy()

return r

class Proxy:

"""Defines a proxy object used to facilitate serialization of a non-serializable object."""

def deproxy(self):

"""Convert the proxy back into the original object."""

raise NotImplementedError()

@dataclass(frozen=True)

class _SimulationLocalConstantsProxy(Proxy):

road_map_spec: Any

road_map_hash: int

def __eq__(self, __o: object) -> bool:

if __o is None:

return False

return self.road_map_hash == getattr(__o, "road_map_hash")

def deproxy(self):

import smarts.sstudio.types

from smarts.core.simulation_local_constants import SimulationLocalConstants

assert isinstance(self.road_map_spec, smarts.sstudio.types.MapSpec)

road_map, _ = self.road_map_spec.builder_fn(self.road_map_spec)

return SimulationLocalConstants(road_map, self.road_map_hash)

def _proxy_slc(v):

return _SimulationLocalConstantsProxy(v.road_map.map_spec, v.road_map_hash)

I looked around, it looks like gymnasium has a potentially better idea for this: https://github.com/Farama-Foundation/Gymnasium/blob/c2a387702c48d2e50f499a4f47d30e293ad75240/gymnasium/utils/ezpickle.py#L4

smarts/core/agent_manager.py

saulfield · 2022-12-15T23:03:13Z

smarts/core/sensor_manager.py

+        # {sensor_id, ...}
+        self._discarded_sensors: Set[str] = set()
+
+    def step(self, sim_frame, renderer):


Type hints would be nice here.

I can add the sim_frame type-hint but not the renderer until we extract an interface.

smarts/core/sensor_manager.py

saulfield · 2022-12-15T23:17:58Z

smarts/core/vehicle_index.py

        """Clean up resources, resetting the index."""
        self._controlled_by = VehicleIndex._build_empty_controlled_by()

        for vehicle in self._vehicles.values():
-            vehicle.teardown(exclude_chassis=True)
+            vehicle.teardown(renderer=renderer, exclude_chassis=True)


Passing the renderer to each of these calls seems a bit surprising. Is there a reasonable way to refactor?

It is difficult. I honestly want to strip out the renderer entirely from the main systems. This is a halfway step towards that.

The end intention is to use the simulation frame to update the state of the renderer on all threads.

smarts/core/vehicle_state.py

smarts/core/vehicle.py

Gamenot · 2023-02-08T01:46:28Z

Changes to be made:

Split up serial and parallel implementations of sensors.
Isolate parallelism to smarts.ray
Use dill serialisation to avoid circular dependency chains
Integrate engine configuration
Reduce parallel implementation code using ray
Extract renderer interface
Add issue for extracting physics

Gamenot · 2023-03-13T14:58:34Z

smarts/core/controllers/action_space_type.py

+class ActionSpaceType(Enum):
+    """Available vehicle action spaces."""


I moved this to its own file to simplify imports.

Gamenot · 2023-03-13T15:01:26Z

smarts/core/plan.py

@@ -72,7 +72,7 @@ def is_specific(self) -> bool:
        """If the goal is reachable at a specific position."""
        return False

-    def is_reached(self, vehicle) -> bool:
+    def is_reached(self, vehicle_state) -> bool:


I tried to restrict passing the vehicle around to data structures because the vehicle has methods that can mutate the engine state.

Gamenot · 2023-03-13T15:06:47Z

smarts/core/plan.py

+    def frame(self) -> PlanFrame:
+        """Get the state of this plan."""
+        assert self._mission
+        return PlanFrame(
+            road_ids=self._route.road_ids if self._route else [], mission=self._mission
+        )
+
+    @classmethod
+    def from_frame(cls, plan_frame: PlanFrame, road_map: RoadMap) -> "Plan":
+        """Generate the plan from a frame."""
+        new_plan = cls(road_map=road_map, mission=plan_frame.mission, find_route=False)
+        new_plan.route = road_map.route_from_road_ids(plan_frame.road_ids)
+        return new_plan


This is an attempt to avoid passing around the road map on simple sets of data like the sensor state. If you need the utility of the plan object you must rebuilt it from the frame and the road map.

Gamenot · 2023-03-13T15:09:59Z

smarts/core/sensors/__init__.py

+class SensorState:
+    """Sensor state information"""
+
+    def __init__(self, max_episode_steps: int, plan_frame: PlanFrame):
+        self._max_episode_steps = max_episode_steps
+        self._plan_frame = plan_frame
+        self._step = 0


This object can then be passed between processes without serialising the road map because of using the plan frame.

Gamenot · 2023-03-13T15:14:54Z

smarts/core/renderer.py

+        try:
+            self.destroy()
+        except TypeError:
+            pass


This silences the program exit race condition error if self is somehow already None.

Wouldn't that throw an AttributeError rather than a TypeError?

Right, it should, I am unsure why I am using TypeError here. The current Renderer class uses Panda3D underneath. I believe it was related to those resources.

Gamenot · 2023-03-13T20:00:23Z

smarts/core/signals.py

@@ -45,7 +45,7 @@ class SignalState(ActorState):

    state: Optional[SignalLightState] = None
    stopping_pos: Optional[Point] = None
-    controlled_lanes: Optional[List[RoadMap.Lane]] = None
+    controlled_lanes: Optional[List[str]] = None


This was to remove road map (via RoadMap.Lane) from signal actor state.

Gamenot · 2023-03-13T20:11:26Z

smarts/core/utils/file.py

 def unpack(obj):
-    """A helper that can be used to print `nestedtuples`. For example,
+    """A helper that can be used to print nested data objects (`tuple`, `dataclass`, `namedtuple`, ...). For example,


This utility ends up being useful for comparison in many cases.

Gamenot · 2023-03-13T20:13:34Z

smarts/core/utils/tests/fixtures.py

@@ -199,7 +199,6 @@ def large_observation():
        ),
        drivable_area_grid_map=DrivableAreaGridMap(
            metadata=GridMapMetadata(
-                created_at=1649853761,
                resolution=0.1953125,


I had to remove created_at because it made the GridMapMetadata object non-deterministic.

Gamenot · 2023-03-13T20:17:11Z

smarts/engine.ini

 [core]
+debug = false
+observation_workers = 0
+reset_retries = 0


Configuration appears to be very easy to add now.

Gamenot · 2023-03-13T20:18:33Z

engine.ini

+[core]
+debug = false
+observation_workers = 2
+reset_retries = 1
+[controllers]


This second config file is used just for testing out separate configuration and will be removed.

smarts/core/agent_manager.py

saulfield · 2023-04-17T16:25:55Z

smarts/core/renderer.py

+        try:
+            self.destroy()
+        except TypeError:
+            pass


Wouldn't that throw an AttributeError rather than a TypeError?

saulfield · 2023-04-17T16:31:21Z

smarts/core/sumo_road_network.py

+        try:
+            junction_check_proc.start()
+        except AssertionError:
+            cls._check_junctions(net_file)


What assertion was being raised?

It has been a while but I believe it was an assertion related to generating a daemon process from a daemon process. This gives a fallback.

saulfield · 2023-04-17T16:47:56Z

smarts/core/tests/test_parallel_sensors.py

+        or serial_total > parallel_2_total
+        or serial_total > parallel_3_total
+        or serial_total > parallel_4_total
+    ), f"{serial_total}, {parallel_1_total}, {parallel_2_total}, {parallel_3_total} {parallel_4_total}"


Would it be useful to add a check for the correctness of the returned observations?

Honestly, I should scrap the time check and just test that the results are the same between the two resolvers.

smarts/core/sensor_manager.py

smarts/core/sensors/__init__.py

smarts/core/simulation_frame.py

smarts/core/smarts.py

Gamenot · 2023-04-17T20:25:53Z

I am going to stop rebasing and switch to merging because the history is now too long.

Gamenot force-pushed the tucker/feature-parallel_observations branch 2 times, most recently from 1f45b2a to a15ca0b Compare November 4, 2022 18:51

Gamenot marked this pull request as ready for review November 28, 2022 20:29

Gamenot changed the title ~~[WIP] Tucker/feature parallel observations~~ Parallel observations Nov 28, 2022

Gamenot linked an issue Nov 29, 2022 that may be closed by this pull request

Parallelize Vehicle Observations #1639

Closed

saulfield reviewed Dec 14, 2022

View reviewed changes

qianyi-sun force-pushed the tucker/feature-parallel_observations branch from dac8642 to 4fa2b58 Compare December 15, 2022 21:53

saulfield reviewed Dec 15, 2022

View reviewed changes

Gamenot force-pushed the tucker/feature-parallel_observations branch 2 times, most recently from 25b75b4 to fc1c9e2 Compare December 30, 2022 20:19

Gamenot added this to the `develop` branch close-down milestone Feb 14, 2023

Gamenot force-pushed the tucker/feature-parallel_observations branch from edd90db to 4482b0c Compare February 16, 2023 19:26

Gamenot changed the base branch from develop to master February 16, 2023 19:27

Gamenot force-pushed the tucker/feature-parallel_observations branch 2 times, most recently from 1da17ea to 55fd879 Compare February 22, 2023 16:21

Gamenot force-pushed the tucker/feature-parallel_observations branch 2 times, most recently from 4021e6e to 431dc5e Compare March 6, 2023 20:43

Gamenot force-pushed the tucker/feature-parallel_observations branch 3 times, most recently from 80a0839 to f4e8634 Compare March 13, 2023 14:57

Gamenot commented Mar 13, 2023

View reviewed changes

Gamenot requested review from saulfield, Adaickalavan and qianyi-sun March 14, 2023 16:00

Gamenot mentioned this pull request Mar 17, 2023

Add vehicle of interest coloring. #1909

Merged

1 task

Gamenot force-pushed the tucker/feature-parallel_observations branch from 6eee8cc to a085b1a Compare March 24, 2023 15:08

Gamenot and others added 8 commits April 11, 2023 10:13

fix tests

7cc8366

Update logging naming.

af2427c

Ensure ray sensor resolver works.

fcee963

Fix warnings and type errors.

e6ba79c

Fix missing init files.

7b442ac

Add headers.

958746a

Add missing docstrings.

fac1237

Fix changed method name.

0c6b139

Gamenot force-pushed the tucker/feature-parallel_observations branch from fb9835c to 0c6b139 Compare April 11, 2023 14:18

make format

ce5aca5

saulfield reviewed Apr 17, 2023

View reviewed changes

Address all comments.

475cc1e

Gamenot and others added 2 commits April 17, 2023 16:31

Merge branch 'master' into tucker/feature-parallel_observations

018219a

Find issue with un-updated sensor.

165a3cb

saulfield approved these changes Apr 18, 2023

View reviewed changes

Gamenot added 11 commits April 20, 2023 13:44

Fix issues.

4dad936

Merge branch 'master' into tucker/feature-parallel_observations

a367a75

Fix base tests syntax.

28cb9f9

Fix test.

7acd5a5

Remove excess prints.

6a6632d

Clean up tests.

dcbc6d7

Futher clean-up of code.

74fd2d2

Remove redundant configuration.

2770644

Install rllib on /examples and /env.

03ec739

Fix argument error.

b22fddb

Fix ray test directory.

f610673

Gamenot merged commit 5e1af42 into master Apr 21, 2023

Gamenot deleted the tucker/feature-parallel_observations branch April 21, 2023 13:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parallel observations #1687

Parallel observations #1687

Gamenot commented Oct 28, 2022

Adaickalavan commented Oct 31, 2022

Gamenot commented Nov 4, 2022 •

edited

Loading

saulfield Dec 14, 2022

Gamenot Jan 4, 2023

Gamenot Jan 6, 2023

saulfield Dec 15, 2022

Gamenot Jan 4, 2023

saulfield Dec 15, 2022

Gamenot Jan 4, 2023 •

edited

Loading

Gamenot Jan 4, 2023

Gamenot commented Feb 8, 2023 •

edited

Loading

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

saulfield Apr 17, 2023

Gamenot Apr 17, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

Gamenot Mar 13, 2023

saulfield Apr 17, 2023

saulfield Apr 17, 2023

Gamenot Apr 17, 2023

saulfield Apr 17, 2023

Gamenot Apr 17, 2023 •

edited

Loading

Gamenot commented Apr 17, 2023

	def dumps(__o):
	"""Serializes the given object."""
	import cloudpickle

	_lazy_init()
	r = __o
	type_ = type(__o)
	# TODO: Add a formatter parameter instead of handling proxies internal to serialization
	proxy_func = _proxies.get(type_)
	if proxy_func:
	r = proxy_func(__o)
	return cloudpickle.dumps(r)


	def loads(__o):
	"""Deserializes the given object."""
	import cloudpickle

	r = cloudpickle.loads(__o)
	if hasattr(r, "deproxy"):
	r = r.deproxy()
	return r


	class Proxy:
	"""Defines a proxy object used to facilitate serialization of a non-serializable object."""

	def deproxy(self):
	"""Convert the proxy back into the original object."""
	raise NotImplementedError()


	@dataclass(frozen=True)
	class _SimulationLocalConstantsProxy(Proxy):
	road_map_spec: Any
	road_map_hash: int

	def __eq__(self, __o: object) -> bool:
	if __o is None:
	return False
	return self.road_map_hash == getattr(__o, "road_map_hash")

	def deproxy(self):
	import smarts.sstudio.types
	from smarts.core.simulation_local_constants import SimulationLocalConstants

	assert isinstance(self.road_map_spec, smarts.sstudio.types.MapSpec)
	road_map, _ = self.road_map_spec.builder_fn(self.road_map_spec)
	return SimulationLocalConstants(road_map, self.road_map_hash)


	def _proxy_slc(v):
	return _SimulationLocalConstantsProxy(v.road_map.map_spec, v.road_map_hash)

		class ActionSpaceType(Enum):
		"""Available vehicle action spaces."""

Parallel observations #1687

Parallel observations #1687

Conversation

Gamenot commented Oct 28, 2022

Adaickalavan commented Oct 31, 2022

Gamenot commented Nov 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gamenot Jan 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gamenot commented Feb 8, 2023 • edited Loading

Changes to be made:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gamenot Apr 17, 2023 • edited Loading

Choose a reason for hiding this comment

Gamenot commented Apr 17, 2023

Gamenot commented Nov 4, 2022 •

edited

Loading

Gamenot Jan 4, 2023 •

edited

Loading

Gamenot commented Feb 8, 2023 •

edited

Loading

Gamenot Apr 17, 2023 •

edited

Loading