Skip to content

Commit

Permalink
Change folder structure for NAS algorithms, rename NASRL to ENAS (kub…
Browse files Browse the repository at this point in the history
…eflow#1143)

* Init commit

* Rename nasrl to enas

* Modify docker ignore

* Undo changes for nas monitor js

* Fix enas cnn docker image name

* Rename ENAS in README
  • Loading branch information
andreyvelich authored and sperlingxx committed Apr 20, 2020
1 parent 8cf9cce commit 9d2ae1d
Show file tree
Hide file tree
Showing 34 changed files with 93 additions and 91 deletions.
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
.gitignore
docs
examples
!examples/v1alpha3/NAS-training-containers/RL-cifar10
!examples/v1alpha3/nas/enas-cnn-cifar10
manifests
pkg/ui/*/frontend/node_modules
pkg/ui/*/frontend/build
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Currently Katib supports the following exploration algorithms:

#### Neural Architecture Search

* [Reinforcement Learning](https://github.com/kubeflow/katib/tree/master/pkg/suggestion/v1alpha3/NAS_Reinforcement_Learning)
* [Efficient Neural Architecture Search (ENAS)](https://github.com/kubeflow/katib/tree/master/pkg/suggestion/v1alpha3/nas/enas)


## Components in Katib
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
FROM python:3.6

RUN if [ "$(uname -m)" = "ppc64le" ]; then \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
pip install cython; \
apt-get -y update && \
apt-get -y install gfortran libopenblas-dev liblapack-dev && \
pip install cython; \
fi
RUN GRPC_HEALTH_PROBE_VERSION=v0.3.1 && \
if [ "$(uname -m)" = "ppc64le" ]; then \
wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-ppc64le; \
else \
wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64; \
fi && \
chmod +x /bin/grpc_health_probe

ADD . /usr/src/app/github.com/kubeflow/katib
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/suggestion/nasrl/v1alpha3
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/suggestion/nas/enas/v1alpha3
RUN pip install --no-cache-dir -r requirements.txt

ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/health/python
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ RUN pip install 'grpcio==1.23.0' 'protobuf==3.9.1' 'googleapis-common-protos==1.
COPY --from=build-env /bin/grpc_health_probe /bin/

ADD . /usr/src/app/github.com/kubeflow/katib
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/suggestion/nasrl/v1alpha3
WORKDIR /usr/src/app/github.com/kubeflow/katib/cmd/suggestion/nas/enas/v1alpha3

ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/v1alpha3/python:/usr/src/app/github.com/kubeflow/katib/pkg/apis/manager/health/python

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@

from pkg.apis.manager.v1alpha3.python import api_pb2_grpc
from pkg.apis.manager.health.python import health_pb2_grpc
from pkg.suggestion.v1alpha3.nasrl_service import NasrlService
from pkg.suggestion.v1alpha3.nas.enas_service import EnasService


_ONE_DAY_IN_SECONDS = 60 * 60 * 24
DEFAULT_PORT = "0.0.0.0:6789"


def serve():
print("NAS RL Suggestion Service")
print("ENAS Suggestion Service")
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
service = NasrlService()
service = EnasService()
api_pb2_grpc.add_SuggestionServicer_to_server(service, server)
health_pb2_grpc.add_HealthServicer_to_server(service, server)
server.add_insecure_port(DEFAULT_PORT)
Expand Down
38 changes: 19 additions & 19 deletions docs/new-algorithm-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The design of Katib follows the `ask-and-tell` pattern:

> They often follow a pattern a bit like this: 1. ask for a new set of parameters 1. walk to the experiment and program in the new parameters 1. observe the outcome of running the experiment 1. walk back to your laptop and tell the optimizer about the outcome 1. go to step 1
When an experiment is created, one algorithm service will be created. Then Katib asks for new sets of parameters via `GetSuggestions` GRPC call. After that, Katib creates new trials according to the sets and observe the outcome. When the trials are finished, Katib tells the metrics of the finished trials to the algorithm, and ask another new sets.
When an experiment is created, one algorithm service will be created. Then Katib asks for new sets of parameters via `GetSuggestions` GRPC call. After that, Katib creates new trials according to the sets and observe the outcome. When the trials are finished, Katib tells the metrics of the finished trials to the algorithm, and ask another new sets.

The new algorithm needs to implement `Suggestion` service defined in [api.proto](../pkg/apis/manager/v1alpha3/api.proto). One sample algorithm looks like:

Expand All @@ -31,8 +31,8 @@ class HyperoptService(
# Convert the experiment in GRPC request to the search space.
# search_space example:
# HyperParameterSearchSpace(
# goal: MAXIMIZE,
# params: [HyperParameter(name: param-1, type: INTEGER, min: 1, max: 5, step: 0),
# goal: MAXIMIZE,
# params: [HyperParameter(name: param-1, type: INTEGER, min: 1, max: 5, step: 0),
# HyperParameter(name: param-2, type: CATEGORICAL, list: cat1, cat2, cat3),
# HyperParameter(name: param-3, type: DISCRETE, list: 3, 2, 6),
# HyperParameter(name: param-4, type: DOUBLE, min: 1, max: 5, step: )]
Expand All @@ -41,41 +41,41 @@ class HyperoptService(
# Convert the trials in GRPC request to the trials in algorithm side.
# trials example:
# [Trial(
# assignment: [Assignment(name=param-1, value=2),
# Assignment(name=param-2, value=cat1),
# Assignment(name=param-3, value=2),
# assignment: [Assignment(name=param-1, value=2),
# Assignment(name=param-2, value=cat1),
# Assignment(name=param-3, value=2),
# Assignment(name=param-4, value=3.44)],
# target_metric: Metric(name="metric-2" value="5643"),
# additional_metrics: [Metric(name=metric-1, value=435),
# target_metric: Metric(name="metric-2" value="5643"),
# additional_metrics: [Metric(name=metric-1, value=435),
# Metric(name=metric-3, value=5643)],
# Trial(
# assignment: [Assignment(name=param-1, value=3),
# Assignment(name=param-2, value=cat2),
# Assignment(name=param-3, value=6),
# Assignment(name=param-4, value=4.44)],
# target_metric: Metric(name="metric-2" value="3242"),
# additional_metrics: [Metric(name=metric=1, value=123),
# target_metric: Metric(name="metric-2" value="3242"),
# additional_metrics: [Metric(name=metric=1, value=123),
# Metric(name=metric-3, value=543)],
trials = Trial.convert(request.trials)
#--------------------------------------------------------------
# Your code here
# Implment the logic to generate new assignments for the given request number.
# For example, if request.request_number is 2, you should return:
# [
# [Assignment(name=param-1, value=3),
# Assignment(name=param-2, value=cat2),
# Assignment(name=param-3, value=3),
# [Assignment(name=param-1, value=3),
# Assignment(name=param-2, value=cat2),
# Assignment(name=param-3, value=3),
# Assignment(name=param-4, value=3.22)
# ],
# [Assignment(name=param-1, value=4),
# Assignment(name=param-2, value=cat4),
# Assignment(name=param-3, value=2),
# [Assignment(name=param-1, value=4),
# Assignment(name=param-2, value=cat4),
# Assignment(name=param-3, value=2),
# Assignment(name=param-4, value=4.32)
# ],
# ]
list_of_assignments = your_logic(search_space, trials, request.request_number)
#--------------------------------------------------------------
# Convert list_of_assignments to
# Convert list_of_assignments to
return api_pb2.GetSuggestionsReply(
trials=Assignment.generate(list_of_assignments)
)
Expand Down Expand Up @@ -223,8 +223,8 @@ Then add a new step in our CI to run the new e2e test case in [test/workflows/co
```diff
// ...
{
name: "run-nasrl-e2e-tests",
template: "run-nasrl-e2e-tests",
name: "run-enas-e2e-tests",
template: "run-enas-e2e-tests",
},
{
name: "run-hyperband-e2e-tests",
Expand Down
8 changes: 4 additions & 4 deletions examples/v1alpha3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,16 +323,16 @@ docker.io/kubeflowkatib/mxnet-mnist
docker.io/kubeflowkatib/pytorch-mnist
```

- Keras cifar10 example for NAS RL with gpu support, [source](https://github.com/kubeflow/katib/blob/master/examples/v1alpha3/NAS-training-containers/RL-cifar10/Dockerfile.cpu).
- Keras cifar10 CNN example for ENAS with gpu support, [source](https://github.com/kubeflow/katib/blob/master/examples/v1alpha3/nas/enas-cnn-cifar10/Dockerfile.gpu).

```
docker.io/kubeflowkatib/nasrl-cifar10-gpu
docker.io/kubeflowkatib/enas-cnn-cifar10-gpu
```

- Keras cifar10 example for NAS RL with cpu support, [source](https://github.com/kubeflow/katib/blob/master/examples/v1alpha3/NAS-training-containers/RL-cifar10/Dockerfile.cpu).
- Keras cifar10 CNN example for ENAS with cpu support, [source](https://github.com/kubeflow/katib/blob/master/examples/v1alpha3/nas/enas-cnn-cifar10/Dockerfile.cpu).

```
docker.io/kubeflowkatib/nasrl-cifar10-cpu
docker.io/kubeflowkatib/enas-cnn-cifar10-cpu
```

- Pytorch operator mnist example, [source](https://github.com/kubeflow/pytorch-operator/blob/master/examples/mnist/mnist.py).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ RUN apt-get update && apt-get install -y software-properties-common && \
wget

ADD . /usr/src/app/github.com/kubeflow/katib
WORKDIR /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/NAS-training-containers/RL-cifar10
WORKDIR /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/nas/enas-cnn-cifar10

RUN pip3 install --upgrade pip
RUN pip3 install --upgrade --no-cache-dir -r requirements-cpu.txt
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/NAS-training-containers/RL-cifar10
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/nas/enas-cnn-cifar10

ENTRYPOINT ["python3", "-u", "RunTrial.py"]
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ RUN apt-get update && apt-get install -y software-properties-common && \


ADD . /usr/src/app/github.com/kubeflow/katib
WORKDIR /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/NAS-training-containers/RL-cifar10
WORKDIR /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/nas/enas-cnn-cifar10

RUN pip3 install --upgrade pip
RUN pip3 install --no-cache-dir -r requirements-gpu.txt
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/NAS-training-containers/RL-cifar10
ENV PYTHONPATH /usr/src/app/github.com/kubeflow/katib/examples/v1alpha3/nas/enas-cnn-cifar10

ENTRYPOINT ["python3.5", "-u", "RunTrial.py"]
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ apiVersion: "kubeflow.org/v1alpha3"
kind: Experiment
metadata:
namespace: kubeflow
name: nas-rl-example-cpu
name: enas-example-cpu
spec:
parallelTrialCount: 2
maxTrialCount: 3
Expand All @@ -22,7 +22,7 @@ spec:
goal: 0.99
objectiveMetricName: Validation-Accuracy
algorithm:
algorithmName: nasrl
algorithmName: enas
trialTemplate:
goTemplate:
rawTemplate: |-
Expand All @@ -36,7 +36,7 @@ spec:
spec:
containers:
- name: {{.Trial}}
image: docker.io/kubeflowkatib/nasrl-cifar10-cpu
image: docker.io/kubeflowkatib/enas-cnn-cifar10-cpu
command:
- "python3.5"
- "-u"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ apiVersion: "kubeflow.org/v1alpha3"
kind: Experiment
metadata:
namespace: kubeflow
name: nas-rl-example-gpu
name: enas-example-gpu
spec:
parallelTrialCount: 3
maxTrialCount: 12
Expand All @@ -19,7 +19,7 @@ spec:
goal: 0.99
objectiveMetricName: Validation-Accuracy
algorithm:
algorithmName: nasrl
algorithmName: enas
trialTemplate:
goTemplate:
rawTemplate: |-
Expand All @@ -33,7 +33,7 @@ spec:
spec:
containers:
- name: {{.Trial}}
image: docker.io/kubeflowkatib/nasrl-cifar10-gpu
image: docker.io/kubeflowkatib/enas-cnn-cifar10-gpu
command:
- "python3.5"
- "-u"
Expand Down
4 changes: 2 additions & 2 deletions manifests/v1alpha3/katib-controller/katib-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ data:
"tpe": {
"image": "gcr.io/kubeflow-images-public/katib/v1alpha3/suggestion-hyperopt"
},
"nasrl": {
"image": "gcr.io/kubeflow-images-public/katib/v1alpha3/suggestion-nasrl",
"enas": {
"image": "gcr.io/kubeflow-images-public/katib/v1alpha3/suggestion-enas",
"imagePullPolicy": "Always",
"resources": {
"limits": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ data:
{{- end}}
{{- end}}
restartPolicy: Never
nasRLCPUTemplate: |-
enasCPUTemplate: |-
apiVersion: batch/v1
kind: Job
metadata:
Expand All @@ -39,7 +39,7 @@ data:
spec:
containers:
- name: {{.Trial}}
image: docker.io/kubeflowkatib/nasrl-cifar10-cpu
image: docker.io/kubeflowkatib/enas-cnn-cifar10-cpu
command:
- "python3.5"
- "-u"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# About the Neural Architecture Search with Reinforcement Learning Suggestion
# About the Efficient Neural Architecture Search

The algorithm follows the idea proposed in _Neural Architecture Search with Reinforcement Learning_ by Zoph & Le (https://arxiv.org/abs/1611.01578), and the implementation is based on the github of _Efficient Neural Architecture Search via Parameter Sharing_ (https://github.com/melodyguan/enas and https://github.com/google-research/google-research/tree/master/enas_lm). It uses a recurrent neural network with LSTM cells as controller to generate neural architecture candidates. And this controller network is updated by policy gradients. However, it currently does not support parameter sharing.
The algorithm follows the idea proposed in _Efficient Neural Architecture Search via Parameter Sharing_ by Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le and Jeff Dean (https://arxiv.org/abs/1802.03268) and _Neural Architecture Search with Reinforcement Learning_ by Barret Zoph and Quoc V. Le (https://arxiv.org/abs/1611.01578).

The implementation is based on the github of _Efficient Neural Architecture Search via Parameter Sharing_ (https://github.com/melodyguan/enas and https://github.com/google-research/google-research/tree/master/enas_lm). It uses a recurrent neural network with LSTM cells as controller to generate neural architecture candidates. And this controller network is updated by policy gradients. However, it currently does not support parameter sharing.

## Definition of a Neural Architecture

Expand Down Expand Up @@ -130,5 +132,5 @@ This neural architecture can be visualized as
2. Add support for recurrent neural networks and build a training container for the Penn Treebank task.
3. Add parameter sharing, if possible.
4. Change LSTM cell from self defined functions in LSTM.py to `tf.nn.rnn_cell.LSTMCell`
5. Store the suggestion checkpoint to PVC to protect against unexpected nasrl service pod restarts
5. Store the suggestion checkpoint to PVC to protect against unexpected enas service pod restarts
6. Add `RequestCount` into API so that the suggestion can clean the information of completed studies.
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@

from pkg.apis.manager.v1alpha3.python import api_pb2
from pkg.apis.manager.v1alpha3.python import api_pb2_grpc
from pkg.suggestion.v1alpha3.NAS_Reinforcement_Learning.Controller import Controller
from pkg.suggestion.v1alpha3.NAS_Reinforcement_Learning.Operation import SearchSpace
from pkg.suggestion.v1alpha3.NAS_Reinforcement_Learning.AlgorithmSettings import parseAlgorithmSettings
from pkg.suggestion.v1alpha3.nas.enas.Controller import Controller
from pkg.suggestion.v1alpha3.nas.enas.Operation import SearchSpace
from pkg.suggestion.v1alpha3.nas.enas.AlgorithmSettings import parseAlgorithmSettings
from pkg.suggestion.v1alpha3.base_health_service import HealthServicer


class NAS_RL_Experiment:
class EnasExperiment:
def __init__(self, request, logger):
self.logger = logger
self.experiment_name = request.experiment.name
Expand Down Expand Up @@ -126,9 +126,9 @@ def print_algorithm_settings(self):
self.logger.info("")


class NasrlService(api_pb2_grpc.SuggestionServicer, HealthServicer):
class EnasService(api_pb2_grpc.SuggestionServicer, HealthServicer):
def __init__(self, logger=None):
super(NasrlService, self).__init__()
super(EnasService, self).__init__()
self.is_first_run = True
self.experiment = None
if logger == None:
Expand Down Expand Up @@ -213,7 +213,7 @@ def SetValidateContextError(self, context, error_message):

def GetSuggestions(self, request, context):
if self.is_first_run:
self.experiment = NAS_RL_Experiment(request, self.logger)
self.experiment = EnasExperiment(request, self.logger)
experiment = self.experiment
if request.request_number > 0:
experiment.num_trials = request.request_number
Expand Down
6 changes: 3 additions & 3 deletions pkg/ui/v1alpha3/frontend/src/reducers/nasCreate.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const initialState = {
commonParametersMetadata: [
{
name: 'Name',
value: 'nasrl-example',
value: 'enas-example',
description: 'A name of an experiment',
},
{
Expand Down Expand Up @@ -50,8 +50,8 @@ const initialState = {
},
],
additionalMetricNames: [],
algorithmName: 'nasrl',
allAlgorithms: ['nasrl'],
algorithmName: 'enas',
allAlgorithms: ['enas'],
algorithmSettings: [
{
name: 'controller_hidden_size',
Expand Down
4 changes: 2 additions & 2 deletions prow_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ workflows:
- cmd/suggestion/chocolate/v1alpha3/*
- cmd/suggestion/hyperband/v1alpha3/*
- cmd/suggestion/hyperopt/v1alpha3/*
- cmd/suggestion/nasrl/v1alpha3/*
- cmd/suggestion/nas/enas/v1alpha3/*
- cmd/suggestion/skopt/v1alpha3/*
- cmd/ui/v1alpha3/*
- examples/v1alpha3/*.yaml
Expand Down Expand Up @@ -69,7 +69,7 @@ workflows:
- cmd/suggestion/chocolate/v1alpha3/*
- cmd/suggestion/hyperband/v1alpha3/*
- cmd/suggestion/hyperopt/v1alpha3/*
- cmd/suggestion/nasrl/v1alpha3/*
- cmd/suggestion/nas/enas/v1alpha3/*
- cmd/suggestion/skopt/v1alpha3/*
- cmd/ui/v1alpha3/*
- examples/v1alpha3/*.yaml
Expand Down
4 changes: 2 additions & 2 deletions scripts/v1alpha3/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ docker build -t ${REGISTRY}/${PREFIX}/suggestion-hyperopt:${TAG} -f ${CMD_PREFIX
docker build -t ${REGISTRY}/${PREFIX}/suggestion-skopt:${TAG} -f ${CMD_PREFIX}/suggestion/skopt/v1alpha3/Dockerfile .
docker build -t ${REGISTRY}/${PREFIX}/suggestion-chocolate:${TAG} -f ${CMD_PREFIX}/suggestion/chocolate/v1alpha3/Dockerfile .
if [ $MACHINE_ARCH == "aarch64" ]; then
docker build -t ${REGISTRY}/${PREFIX}/suggestion-nasrl:${TAG} -f ${CMD_PREFIX}/suggestion/nasrl/v1alpha3/Dockerfile.aarch64 .
docker build -t ${REGISTRY}/${PREFIX}/suggestion-enas:${TAG} -f ${CMD_PREFIX}/suggestion/nas/enas/v1alpha3/Dockerfile.aarch64 .
else
docker build -t ${REGISTRY}/${PREFIX}/suggestion-nasrl:${TAG} -f ${CMD_PREFIX}/suggestion/nasrl/v1alpha3/Dockerfile .
docker build -t ${REGISTRY}/${PREFIX}/suggestion-enas:${TAG} -f ${CMD_PREFIX}/suggestion/nas/enas/v1alpha3/Dockerfile .
fi
docker build -t ${REGISTRY}/${PREFIX}/suggestion-hyperband:${TAG} -f ${CMD_PREFIX}/suggestion/hyperband/v1alpha3/Dockerfile .
Loading

0 comments on commit 9d2ae1d

Please sign in to comment.