Skip to content
This repository was archived by the owner on Dec 11, 2022. It is now read-only.

Commit c973828

Browse files
galnovgalnov
authored
Require Python 3.6 + Changes to CI configuration (#452)
* Change build_*_env jobs to pull base image of current "tag" instead of "master" image * Change nightly flow so build_*_env jobs now gated by build_base (so change in previous bullet works in nightly) * Bugfix in CheckpointDataStore: Call to object.__init__ with parameters * Disabling unstable Doom A3C and ACER golden tests
1 parent a6689b6 commit c973828

File tree

7 files changed

+46
-34
lines changed

7 files changed

+46
-34
lines changed

‎.circleci/config.yml

+25-15
Original file line numberDiff line numberDiff line change
@@ -75,9 +75,9 @@ jobs:
7575
command: |
7676
REGISTRY=316971102342.dkr.ecr.us-west-2.amazonaws.com
7777
TAG=$(git describe --tags --always --dirty)
78-
docker pull ${REGISTRY}/coach-base:${MASTER_BRANCH}
79-
docker tag ${REGISTRY}/coach-base:${MASTER_BRANCH} coach-base:master
80-
docker build --cache-from ${REGISTRY}/coach-base:${MASTER_BRANCH} -t ${REGISTRY}/coach-gym_environment:${TAG} -f docker/Dockerfile.gym_environment .
78+
docker pull ${REGISTRY}/coach-base:${TAG}
79+
docker tag ${REGISTRY}/coach-base:${TAG} coach-base:master
80+
docker build --cache-from ${REGISTRY}/coach-base:${TAG} -t ${REGISTRY}/coach-gym_environment:${TAG} -f docker/Dockerfile.gym_environment .
8181
docker push ${REGISTRY}/coach-gym_environment:${TAG}
8282
no_output_timeout: 10m
8383

@@ -93,9 +93,9 @@ jobs:
9393
command: |
9494
REGISTRY=316971102342.dkr.ecr.us-west-2.amazonaws.com
9595
TAG=$(git describe --tags --always --dirty)
96-
docker pull ${REGISTRY}/coach-base:${MASTER_BRANCH}
97-
docker tag ${REGISTRY}/coach-base:${MASTER_BRANCH} coach-base:master
98-
docker build --cache-from ${REGISTRY}/coach-base:${MASTER_BRANCH} -t ${REGISTRY}/coach-doom_environment:${TAG} -f docker/Dockerfile.doom_environment .
96+
docker pull ${REGISTRY}/coach-base:${TAG}
97+
docker tag ${REGISTRY}/coach-base:${TAG} coach-base:master
98+
docker build --cache-from ${REGISTRY}/coach-base:${TAG} -t ${REGISTRY}/coach-doom_environment:${TAG} -f docker/Dockerfile.doom_environment .
9999
docker push ${REGISTRY}/coach-doom_environment:${TAG}
100100
no_output_timeout: 10m
101101

@@ -111,9 +111,9 @@ jobs:
111111
command: |
112112
REGISTRY=316971102342.dkr.ecr.us-west-2.amazonaws.com
113113
TAG=$(git describe --tags --always --dirty)
114-
docker pull ${REGISTRY}/coach-base:${MASTER_BRANCH}
115-
docker tag ${REGISTRY}/coach-base:${MASTER_BRANCH} coach-base:master
116-
docker build --cache-from ${REGISTRY}/coach-base:${MASTER_BRANCH} --build-arg MUJOCO_KEY=${MUJOCO_KEY} -t ${REGISTRY}/coach-mujoco_environment:${TAG} -f docker/Dockerfile.mujoco_environment .
114+
docker pull ${REGISTRY}/coach-base:${TAG}
115+
docker tag ${REGISTRY}/coach-base:${TAG} coach-base:master
116+
docker build --cache-from ${REGISTRY}/coach-base:${TAG} --build-arg MUJOCO_KEY=${MUJOCO_KEY} -t ${REGISTRY}/coach-mujoco_environment:${TAG} -f docker/Dockerfile.mujoco_environment .
117117
docker push ${REGISTRY}/coach-mujoco_environment:${TAG}
118118
no_output_timeout: 10m
119119

@@ -130,9 +130,9 @@ jobs:
130130
# command: |
131131
# REGISTRY=316971102342.dkr.ecr.us-west-2.amazonaws.com
132132
# TAG=$(git describe --tags --always --dirty)
133-
# docker pull ${REGISTRY}/coach-base:${MASTER_BRANCH}
134-
# docker tag ${REGISTRY}/coach-base:${MASTER_BRANCH} coach-base:master
135-
# docker build --cache-from ${REGISTRY}/coach-base:${MASTER_BRANCH} -t ${REGISTRY}/coach-starcraft_environment:${TAG} -f docker/Dockerfile.starcraft_environment .
133+
# docker pull ${REGISTRY}/coach-base:${TAG}
134+
# docker tag ${REGISTRY}/coach-base:${TAG} coach-base:master
135+
# docker build --cache-from ${REGISTRY}/coach-base:${TAG} -t ${REGISTRY}/coach-starcraft_environment:${TAG} -f docker/Dockerfile.starcraft_environment .
136136
# docker push ${REGISTRY}/coach-starcraft_environment:${TAG}
137137
# no_output_timeout: 10m
138138

@@ -614,6 +614,9 @@ workflows:
614614
- multinode_test:
615615
requires:
616616
- multinode_approval
617+
# NOTE: build_gym/doom/mujoco_env MUST occur after successful build_base stage
618+
# In this workflow this is satisfied by having this flow:
619+
# build_base --> e2e_approval --> build_*_env
617620
- e2e_approval:
618621
type: approval
619622
requires:
@@ -718,10 +721,17 @@ workflows:
718721
only:
719722
- master
720723
jobs:
721-
- build_gym_env
722-
- build_doom_env
723-
- build_mujoco_env
724724
- build_base
725+
# NOTE: build_gym/doom/mujoco_env MUST occur after successful build_base stage
726+
- build_gym_env:
727+
requires:
728+
- build_base
729+
- build_doom_env:
730+
requires:
731+
- build_base
732+
- build_mujoco_env:
733+
requires:
734+
- build_base
725735
- unit_tests:
726736
requires:
727737
- build_base

‎docker/Dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ FROM coach-base:master as builder
22

33
# prep some of the more common environments
44
# Gym (installed with coach)
5-
Run pip3 install gym[atari]==0.12.5 box2d
5+
RUN pip3 install gym[atari]==0.12.5 box2d
66
# Mujoco
77
RUN mkdir -p ~/.mujoco \
88
&& wget https://www.roboti.us/download/mjpro150_linux.zip -O mujoco.zip \

‎docker/Dockerfile.base

+10
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,16 @@ RUN apt-get update && \
2929
apt-get clean autoclean && \
3030
apt-get autoremove -y
3131

32+
################################
33+
# Install Python 3.6 #
34+
################################
35+
36+
RUN DEBIAN_FRONTEND=noninteractive add-apt-repository --yes ppa:deadsnakes/ppa && apt-get update
37+
RUN DEBIAN_FRONTEND=noninteractive apt-get install --yes python3.6-dev python3.6 python3-pip
38+
39+
RUN rm /usr/bin/python3
40+
RUN ln -s /usr/bin/python3.6 /usr/bin/python3
41+
3242
############################
3343
# Install Pip Requirements #
3444
############################

‎rl_coach/data_stores/checkpoint_data_store.py

-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@ class CheckpointDataStore(object):
2525
A DataStore which relies on the GraphManager check pointing methods to communicate policies.
2626
"""
2727
def __init__(self, *args, **kwargs):
28-
super().__init__(*args, **kwargs)
2928
self.checkpoint_num = 0
3029

3130
def end_of_policies(self) -> bool:

‎rl_coach/presets/Doom_Basic_A3C.py

+4-4
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,10 @@
4242
# Test #
4343
########
4444
preset_validation_params = PresetValidationParameters()
45-
preset_validation_params.test = True
46-
preset_validation_params.min_reward_threshold = 20
47-
preset_validation_params.max_episodes_to_achieve_reward = 400
48-
preset_validation_params.num_workers = 8
45+
# preset_validation_params.test = True
46+
# preset_validation_params.min_reward_threshold = 20
47+
# preset_validation_params.max_episodes_to_achieve_reward = 400
48+
# preset_validation_params.num_workers = 8
4949

5050

5151
graph_manager = BasicRLGraphManager(agent_params=agent_params, env_params=env_params,

‎rl_coach/presets/Doom_Basic_ACER.py

+4-4
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,10 @@
4444
# Test #
4545
########
4646
preset_validation_params = PresetValidationParameters()
47-
preset_validation_params.test = True
48-
preset_validation_params.min_reward_threshold = 20
49-
preset_validation_params.max_episodes_to_achieve_reward = 400
50-
preset_validation_params.num_workers = 8
47+
# preset_validation_params.test = True
48+
# preset_validation_params.min_reward_threshold = 20
49+
# preset_validation_params.max_episodes_to_achieve_reward = 400
50+
# preset_validation_params.num_workers = 8
5151

5252

5353
graph_manager = BasicRLGraphManager(agent_params=agent_params, env_params=env_params,

‎setup.py

+2-9
Original file line numberDiff line numberDiff line change
@@ -66,14 +66,7 @@
6666

6767
if not using_GPU:
6868
if not slim_package:
69-
# For linux wth no GPU, we install the Intel optimized version of TensorFlow
70-
if sys.platform == "linux" or sys.platform == "linux2":
71-
# CI: limiting version to 1.13.1 due to
72-
# https://github.com/tensorflow/tensorflow/issues/29617
73-
# (reproduced with intel-tensorflow 1.14.0 but not with 1.13.1)
74-
install_requires.append('intel-tensorflow==1.13.1')
75-
else:
76-
install_requires.append('tensorflow>=1.9.0,<=1.14.0')
69+
install_requires.append('tensorflow>=1.9.0,<=1.14.0')
7770
extras['mxnet'] = ['mxnet-mkl>=1.3.0']
7871
else:
7972
if not slim_package:
@@ -94,7 +87,7 @@
9487
author='Intel AI Lab',
9588
author_email='[email protected]',
9689
packages=find_packages(),
97-
python_requires=">=3.5.*",
90+
python_requires=">=3.6.*",
9891
install_requires=install_requires,
9992
extras_require=extras,
10093
package_data={'rl_coach': ['dashboard_components/*.css',

0 commit comments

Comments
 (0)