3. Getting Started

A quick guide for simulating robots helping people with Assistive Gym.

1. Installation

We will first want to install Assistive Gym and dependencies.
If you prefer to install using a python virtual environment, see the Install Guide.

python3 -m pip install --upgrade pip
git clone https://github.com/Healthcare-Robotics/assistive-gym.git
cd assistive-gym
python3 -m pip install -e .

2. Test out a few environments

We can quickly visualize some of the existing assistive robotics environments using env_viewer.py.
Here, a Sawyer robot will help clean off a person's arm who is lying in a resting pose on a bed.

python3 -m assistive_gym --env "BedBathingSawyer-v1"

We can also visualize a collaborative assistance environment where both the robot and human take random actions.

python3 -m assistive_gym --env "ScratchItchJacoHuman-v1"

When creating new environments, this viewer can also be helpful for visualizing and debugging changes.

3. Running a pretrained control policy

NOTE: Pretrained policies are not yet available for the recent v1.0 of Assistive Gym. In order to use and evaluate with pretrained robot control policies, please install the v0.1 branch of Assistive Gym.

We provide trained robot/human control policies for all of the available environments.
These control policies were trained using Proximal Policy Optimization (PPO) through a PyTorch library.
If you do not have wget installed on your machine, you can download the models directly from the GitHub release page.

Downloading policies and libraries

python3 -m pip install git+https://github.com/Zackory/pytorch-a2c-ppo-acktr --no-cache-dir
python3 -m pip install git+https://github.com/openai/baselines.git
# Download pretrained policies
wget -O trained_models/ppo/pretrained_policies.zip https://github.com/Healthcare-Robotics/assistive-gym/releases/download/0.100/pretrained_policies.zip
unzip trained_models/ppo/pretrained_policies.zip -d trained_models/ppo

Here we will run one of these policies for a PR2 providing drinking assistance to a person who has learned to actively collaborate with the robot through co-optimization.

python3 -m ppo.enjoy_coop --env-name "DrinkingPR2Human-v0"

4. Train a new control policy

New control policies for Assistive Gym can be trained using any library that works with the OpenAI Gym interface.
Here, we can train a new policy for a Jaco robot helping to scratch an itch using the PyTorch library from above.

python3 -m assistive_gym.learn --env "ScratchItchJaco-v1" --algo ppo --train --train-timesteps 20000 --save-dir ./trained_models_new/

As an example, we will only train for 20,000 time steps (100 simulation rollouts), yet increasing num-env-steps can greatly improve the learned policy. The pretrained policies we provide were each trained using 10,000,000 time steps (50,000 simulation rollouts).
We can now visualize the newly learned control policy:

python3 -m assistive_gym.learn --env "ScratchItchJaco-v1" --algo ppo --render --seed 0 --load-policy-path ./trained_models_new/

5. Next Steps

See the list of common commands in Assistive Gym

Check out additional examples on the Google Colab Wiki page.