Added RollBall env

[guru-narayana]

A simple task where the objective is to push and roll a ball to a goal region at the other end of the table.
When testing with baseline PPO please use max_steps of 60, the ball takes time to roll

[StoneT2000]

Can you provide the exact command line script for ppo that I can just copy and paste?

Thank you.

[guru-narayana]

Updated everything in accordance with your recommendations.

Use the following command to run PPO

python examples/baselines/ppo/ppo.py --env_id="RollBall-v1"  --num_envs=1024 --update_epochs=8 --num_minibatches=32 --seed=100 --total_timesteps=100_00_000 --eval_freq=8 --num-steps=60 --num_eval_steps=60 --gamma 0.95

[StoneT2000]

I just checked the video. how did you generate it? It looks like the first frame is from another episode (asking about how you generated since this could be a bug with a tool of ours if you used our tools)

[guru-narayana]

I think I found where bug could be, when I am saving the trajectory from PPO with "--evaluate" its reset params is empty so that could be root cause. Is there a way to prevent this ?

[StoneT2000]

Ah so you are straight running the PPO code in the repo. I will check this

[StoneT2000]

I am still training the environment @guru-narayana to check it solves in a reasonalbe time (the PPO script uses 100M steps which is a lot. If it solves in about an hour on a 3080 I will merge this in anyway).

I recommend trying to tune the reward function a bit more (maybe a staged reward function can work better). Another point is that the reward gets the agent to be 0.05m behind the ball, but this can be suboptimal as you need to hit the ball at an angle.

[StoneT2000]

Actually for the example ppo script did you mean to write 100_000_000, it says 100_00_000

[guru-narayana]

I am still training the environment @guru-narayana to check it solves in a reasonalbe time (the PPO script uses 100M steps which is a lot. If it solves in about an hour on a 3080 I will merge this in anyway).

I recommend trying to tune the reward function a bit more (maybe a staged reward function can work better). Another point is that the reward gets the agent to be 0.05m behind the ball, but this can be suboptimal as you need to hit the ball at an angle.

I made suggested modification and now the agent needs to be at 0.05m behind the ball and along the direction of the goal to get reward. I also made the reward function staged.

please use this modified command to test the environment.
python ManiSkill/examples/baselines/ppo/ppo.py --env_id="RollBall-v1" --num_eval_envs=8 --num_envs=1024 --update_epochs=8 --num_minibatches=32 --total_timesteps=20_000_000 --eval_freq=10 --num-steps=80 --num_eval_steps=80 --gamma=0.95

[guru-narayana]

Actually for the example ppo script did you mean to write 100_000_000, it says 100_00_000

100_00_000 was correct previously, but now please use the command in my recent comment for execution.

[StoneT2000]

Furthermore can you merge in the main branch? It seems you may have used a version on the main branch that had a small bug with the ManiSkillVectorEnv (apoligies for that). Things should run faster now / correctly. Otherwise i can verify this task works correctly, only that one small concern about the need to have a delayed boolean

[guru-narayana]

Furthermore can you merge in the main branch? It seems you may have used a version on the main branch that had a small bug with the ManiSkillVectorEnv (apoligies for that). Things should run faster now / correctly. Otherwise i can verify this task works correctly, only that one small concern about the need to have a delayed boolean

Done