The question about running baselines/marl_benchmark program

[JasonBourne1998]

High Level Description
[I tried to explore marl algorithm, but unfortunately, if I follow the instrction of readme, I can't run the whole program. It seems that there is a problem with the call of some libraries. I debugged it, but I still can't solve it. Could you please help me.]

Desired SMARTS version
[0.6]

Operating System
[ ubuntu 20.04]

Problems
[some import libraries could not be called if following the instrction https://github.com/huawei-noah/SMARTS/tree/master/baselines/marl_benchmark]

[Gamenot]

Hello @JasonBourne1998, I am not sure I can help you with just what you have provided. But, I will take a look at the benchmark tomorrow.

[Gamenot]

It looks like the issue likely has to do with the version of open-cv needing to be at "opencv-python<=4.1.2.30" to prevent the open-cv error with ray[rllib].

[Gamenot]

Work towards fixes: #1126

[JasonBourne1998]

Work towards fixes: #1126

Thanks for your reply, I have successfully run the PPO algorithm, but it seems that maddpg algorithm is stuck on 'rllib' part. In addition, it seems that the code group has not updated readme and code architecture. It seems that the latest version of the code can't run, so I have to switch to the code of 0.4.7 to run it. However, this is not important. The key for me now is that I can't run the maddpg algorithm. The relevant errors are attached below. Could you please help me see the causes of this error? It seems that the config parameter is not properly considered. When maddpg runs, the corresponding early in yaml file_ The parameters of the done script do not match the script?
@Gamenot

Failure # 1 (occurred at 2022-05-03_23-16-04)
Traceback (most recent call last):
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 726, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 489, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/worker.py", line 1452, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(KeyError): �[36mray::MADDPG2.train()�[39m (pid=92767, ip=192.168.1.86)
File "python/ray/_raylet.pyx", line 443, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 477, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 481, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 106, in init
Trainer.init(self, config, env, logger_creator)
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 477, in init
super().init(config, logger_creator)
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/tune/trainable.py", line 249, in init
self.setup(copy.deepcopy(self.config))
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 630, in setup
self._init(self.config, self.env_creator)
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 138, in _init
num_workers=self.config["num_workers"])
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 707, in _make_workers
logdir=self.logdir)
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/evaluation/worker_set.py", line 81, in init
lambda p, pid: (pid, p.observation_space, p.action_space)))
ray.exceptions.RayTaskError(KeyError): �[36mray::RolloutWorker.foreach_policy()�[39m (pid=92760, ip=192.168.1.86)
File "python/ray/_raylet.pyx", line 443, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 477, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 481, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 366, in init
self.env = _validate_env(env_creator(env_context))
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 1187, in
register_env(name, lambda config: env_object(config))
File "/home/jojo/anaconda3/envs/tf2x/lib/python3.7/site-packages/benchmark/wrappers/rllib/frame_stack.py", line 83, in init
config = config["custom_config"]
KeyError: 'custom_config'

[Gamenot]

Hello @JasonBourne1998, I have patched together a 0.4.7 branch that works with the marl benchmark. I have put up a branch marl_benchmark_0_4_7(tagged `marl_0_4_7) and updated the instructions for that branch.

[Gamenot]

Hello @JasonBourne1998, I have patched together a 0.4.7 branch that works with the marl benchmark. I have put up a branch marl_benchmark_0_4_7(tagged `marl_0_4_7) and updated the instructions for that branch.

I tested it with maddpg and it appeared to work without the bug you mention.