From fbbe14d814fb3fe4bd85fff143a4b7c55cccc4f8 Mon Sep 17 00:00:00 2001 From: kvartet <48014605+kvartet@users.noreply.github.com> Date: Mon, 21 Dec 2020 12:33:04 +0800 Subject: [PATCH 1/5] Update README (#3215) --- README.md | 164 +++++++++++++++++++++++++++--------------------------- 1 file changed, 82 insertions(+), 82 deletions(-) diff --git a/README.md b/README.md index 6f813a6a7e..2e68e38653 100644 --- a/README.md +++ b/README.md @@ -14,9 +14,9 @@ [简体中文](README_zh_CN.md) -**NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate** Feature Engineering, Neural Architecture Search, Hyperparameter Tuning and Model Compression. +**NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate** Feature Engineering, Neural Architecture Search, Hyperparameter Tuning and Model Compression. -The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like Local Machine, Remote Servers, OpenPAI, Kubeflow, FrameworkController on K8S (AKS etc.), DLWorkspace (aka. DLTS), AML (Azure Machine Learning), AdaptDL (aka. ADL) and other cloud options. +The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like Local Machine, Remote Servers, OpenPAI, Kubeflow, FrameworkController on K8S (AKS etc.), DLWorkspace (aka. DLTS), AML (Azure Machine Learning), AdaptDL (aka. ADL) and other cloud options. ## **Who should consider using NNI** @@ -68,7 +68,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
  • TensorFlow
  • MXNet
  • Caffe2
    • - More...
    + More...
    - Hyperparameter Tuning + Hyperparameter Tuning - Neural Architecture Search + Neural Architecture Search - Model Compression + Model Compression - Feature Engineering (Beta) + Feature Engineering (Beta) - Early Stop Algorithms + Early Stop Algorithms @@ -193,21 +193,21 @@ Within the following table, we summarized the current NNI capabilities, we are g @@ -239,8 +239,8 @@ For detail system requirements of NNI, please refer to [here](https://nni.readth Note: * If there is any privilege issue, add `--user` to install NNI in the user directory. -* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install [NNI on Windows](docs/en_US/Tutorial/InstallationWin.md). -* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.md). For FAQ on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/InstallationWin.md#faq). +* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install [NNI on Windows](docs/en_US/Tutorial/InstallationWin.rst). +* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.rst). For FAQ on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/InstallationWin.rst#faq). ### **Verify installation** @@ -294,7 +294,7 @@ You can use these commands to get more information about the experiment ----------------------------------------------------------------------- ``` -* Open the `Web UI url` in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. [Here](docs/en_US/Tutorial/WebUI.md) are more Web UI pages. +* Open the `Web UI url` in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. [Here](docs/en_US/Tutorial/WebUI.rst) are more Web UI pages. @@ -316,14 +316,14 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope After getting familiar with contribution agreements, you are ready to create your first PR =), follow the NNI developer tutorials to get start: -* We recommend new contributors to start with simple issues: ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22). -* [NNI developer environment installation tutorial](docs/en_US/Tutorial/SetupNniDeveloperEnvironment.md) -* [How to debug](docs/en_US/Tutorial/HowToDebug.md) -* If you have any questions on usage, review [FAQ](https://github.com/microsoft/nni/blob/master/docs/en_US/Tutorial/FAQ.md) first, if there are no relevant issues and answers to your question, try contact NNI dev team and users in [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) or [File an issue](https://github.com/microsoft/nni/issues/new/choose) on GitHub. -* [Customize your own Tuner](docs/en_US/Tuner/CustomizeTuner.md) -* [Implement customized TrainingService](docs/en_US/TrainingService/HowToImplementTrainingService.md) -* [Implement a new NAS trainer on NNI](docs/en_US/NAS/Advanced.md) -* [Customize your own Advisor](docs/en_US/Tuner/CustomizeAdvisor.md) +* We recommend new contributors to start with simple issues: [good first issue](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or [help-wanted](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22). +* [NNI developer environment installation tutorial](docs/en_US/Tutorial/SetupNniDeveloperEnvironment.rst) +* [How to debug](docs/en_US/Tutorial/HowToDebug.rst) +* If you have any questions on usage, review [FAQ](https://github.com/microsoft/nni/blob/master/docs/en_US/Tutorial/FAQ.rst) first, if there are no relevant issues and answers to your question, try contact NNI dev team and users in [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) or [File an issue](https://github.com/microsoft/nni/issues/new/choose) on GitHub. +* [Customize your own Tuner](docs/en_US/Tuner/CustomizeTuner.rst) +* [Implement customized TrainingService](docs/en_US/TrainingService/HowToImplementTrainingService.rst) +* [Implement a new NAS trainer on NNI](docs/en_US/NAS/Advanced.rst) +* [Customize your own Advisor](docs/en_US/Tuner/CustomizeAdvisor.rst) ## **External Repositories and References** With authors' permission, we listed a set of NNI usage examples and relevant articles. @@ -331,14 +331,14 @@ With authors' permission, we listed a set of NNI usage examples and relevant art * ### **External Repositories** ### * Run [ENAS](examples/nas/enas/README.md) with NNI * [Automatic Feature Engineering](examples/feature_engineering/auto-feature-engineering/README.md) with NNI - * [Hyperparameter Tuning for Matrix Factorization](https://github.com/microsoft/recommenders/blob/master/notebooks/04_model_select_and_optimize/nni_surprise_svd.ipynb) with NNI + * [Hyperparameter Tuning for Matrix Factorization](https://github.com/microsoft/recommenders/blob/master/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb) with NNI * [scikit-nni](https://github.com/ksachdeva/scikit-nni) Hyper-parameter search for scikit-learn pipelines using NNI * ### **Relevant Articles** ### - * [Hyper Parameter Optimization Comparison](docs/en_US/CommunitySharings/HpoComparison.md) - * [Neural Architecture Search Comparison](docs/en_US/CommunitySharings/NasComparison.md) - * [Parallelizing a Sequential Algorithm TPE](docs/en_US/CommunitySharings/ParallelizingTpeSearch.md) - * [Automatically tuning SVD with NNI](docs/en_US/CommunitySharings/RecommendersSvd.md) - * [Automatically tuning SPTAG with NNI](docs/en_US/CommunitySharings/SptagAutoTune.md) + * [Hyper Parameter Optimization Comparison](docs/en_US/CommunitySharings/HpoComparison.rst) + * [Neural Architecture Search Comparison](docs/en_US/CommunitySharings/NasComparison.rst) + * [Parallelizing a Sequential Algorithm TPE](docs/en_US/CommunitySharings/ParallelizingTpeSearch.rst) + * [Automatically tuning SVD with NNI](docs/en_US/CommunitySharings/RecommendersSvd.rst) + * [Automatically tuning SPTAG with NNI](docs/en_US/CommunitySharings/SptagAutoTune.rst) * [Find thy hyper-parameters for scikit-learn pipelines using Microsoft NNI](https://towardsdatascience.com/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1) * **Blog (in Chinese)** - [AutoML tools (Advisor, NNI and Google Vizier) comparison](http://gaocegege.com/Blog/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/katib-new#%E6%80%BB%E7%BB%93%E4%B8%8E%E5%88%86%E6%9E%90) by [@gaocegege](https://github.com/gaocegege) - 总结与分析 section of design and implementation of kubeflow/katib * **Blog (in Chinese)** - [A summary of NNI new capabilities in 2019](https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA) by @squirrelsc @@ -356,7 +356,7 @@ Join IM discussion groups: ## Related Projects -Targeting at openness and advancing state-of-art technology, [Microsoft Research (MSR)](https://www.microsoft.com/en-us/research/group/systems-research-group-asia/) had also released few other open source projects. +Targeting at openness and advancing state-of-art technology, [Microsoft Research (MSR)](https://www.microsoft.com/en-us/research/group/systems-and-networking-research-group-asia/) had also released few other open source projects. * [OpenPAI](https://github.com/Microsoft/pai) : an open source platform that provides complete AI model training and resource management capabilities, it is easy to extend and supports on-premise, cloud and hybrid environments in various scale. * [FrameworkController](https://github.com/Microsoft/frameworkcontroller) : an open source general-purpose Kubernetes Pod Controller that orchestrate all kinds of applications on Kubernetes by a single controller. From ab7f469393403ee48af18bbfb90fdca928f15021 Mon Sep 17 00:00:00 2001 From: Shinai Yang Date: Mon, 4 Jan 2021 12:05:16 +0800 Subject: [PATCH 2/5] remove paiYarn --- docs/en_US/TrainingService/PaiYarnMode.rst | 193 ------------------ .../TrainingService/RemoteMachineMode.rst | 7 + docs/en_US/training_services.rst | 1 - examples/trials/auto-gbdt/config_paiYarn.yml | 32 --- .../trials/cifar10_pytorch/config_paiYarn.yml | 32 --- .../trials/efficientnet/config_paiYarn.yml | 28 --- examples/trials/ga_squad/config_paiYarn.yml | 32 --- .../trials/mnist-advisor/config_paiYarn.yml | 38 ---- .../mnist-annotation/config_paiYarn.yml | 31 --- .../mnist-batch-tune-keras/config_paiYarn.yml | 29 --- .../trials/mnist-keras/config_paiYarn.yml | 32 --- .../trials/mnist-pytorch/config_paiYarn.yml | 32 --- examples/trials/mnist-tfv1/config_paiYarn.yml | 32 --- .../FashionMNIST/config_paiYarn.yml | 39 ---- .../cifar10/config_paiYarn.yml | 39 ---- .../sklearn/classification/config_paiYarn.yml | 32 --- .../sklearn/regression/config_paiYarn.yml | 32 --- 17 files changed, 7 insertions(+), 654 deletions(-) delete mode 100644 docs/en_US/TrainingService/PaiYarnMode.rst delete mode 100644 examples/trials/auto-gbdt/config_paiYarn.yml delete mode 100644 examples/trials/cifar10_pytorch/config_paiYarn.yml delete mode 100644 examples/trials/efficientnet/config_paiYarn.yml delete mode 100644 examples/trials/ga_squad/config_paiYarn.yml delete mode 100644 examples/trials/mnist-advisor/config_paiYarn.yml delete mode 100644 examples/trials/mnist-annotation/config_paiYarn.yml delete mode 100644 examples/trials/mnist-batch-tune-keras/config_paiYarn.yml delete mode 100644 examples/trials/mnist-keras/config_paiYarn.yml delete mode 100644 examples/trials/mnist-pytorch/config_paiYarn.yml delete mode 100644 examples/trials/mnist-tfv1/config_paiYarn.yml delete mode 100644 examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml delete mode 100644 examples/trials/network_morphism/cifar10/config_paiYarn.yml delete mode 100644 examples/trials/sklearn/classification/config_paiYarn.yml delete mode 100644 examples/trials/sklearn/regression/config_paiYarn.yml diff --git a/docs/en_US/TrainingService/PaiYarnMode.rst b/docs/en_US/TrainingService/PaiYarnMode.rst deleted file mode 100644 index 34f3eba643..0000000000 --- a/docs/en_US/TrainingService/PaiYarnMode.rst +++ /dev/null @@ -1,193 +0,0 @@ -.. role:: raw-html(raw) - :format: html - - -**Run an Experiment on OpenpaiYarn** -======================================== - -The original ``pai`` mode is modificated to ``paiYarn`` mode, which is a distributed training platform based on Yarn. - -Setup environment ------------------ - -Install NNI, follow the install guide `here <../Tutorial/QuickStart.rst>`__. - -Run an experiment ------------------ - -Use ``examples/trials/mnist-tfv1`` as an example. The NNI config YAML file's content is like: - -.. code-block:: yaml - - authorName: your_name - experimentName: auto_mnist - # how many trials could be concurrently running - trialConcurrency: 2 - # maximum experiment running duration - maxExecDuration: 3h - # empty means never stop - maxTrialNum: 100 - # choice: local, remote, pai, paiYarn - trainingServicePlatform: paiYarn - # search space file - searchSpacePath: search_space.json - # choice: true, false - useAnnotation: false - tuner: - builtinTunerName: TPE - classArgs: - optimize_mode: maximize - trial: - command: python3 mnist.py - codeDir: ~/nni/examples/trials/mnist-tfv1 - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - image: msranni/nni:latest - # Configuration to access OpenpaiYarn Cluster - paiYarnConfig: - userName: your_paiYarn_nni_user - passWord: your_paiYarn_password - host: 10.1.1.1 - -Note: You should set ``trainingServicePlatform: paiYarn`` in NNI config YAML file if you want to start experiment in paiYarn mode. - -Compared with `LocalMode `__ and `RemoteMachineMode `__\ , trial configuration in paiYarn mode have these additional keys: - - -* cpuNum - - * Required key. Should be positive number based on your trial program's CPU requirement - -* memoryMB - - * Required key. Should be positive number based on your trial program's memory requirement - -* image - - * Required key. In paiYarn mode, your trial program will be scheduled by OpenpaiYarn to run in `Docker container `__. This key is used to specify the Docker image used to create the container in which your trial will run. - * We already build a docker image :githublink:`nnimsra/nni `. You can either use this image directly in your config file, or build your own image based on it. - -* virtualCluster - - * Optional key. Set the virtualCluster of OpenpaiYarn. If omitted, the job will run on default virtual cluster. - -* shmMB - - * Optional key. Set the shmMB configuration of OpenpaiYarn, it set the shared memory for one task in the task role. - -* authFile - - * Optional key, Set the auth file path for private registry while using paiYarn mode, `Refer `__\ , you can prepare the authFile and simply provide the local path of this file, NNI will upload this file to HDFS for you. - -* - portList - - - * - Optional key. Set the portList configuration of OpenpaiYarn, it specifies a list of port used in container, `Refer `__.\ :raw-html:`
    ` - The config schema in NNI is shown below: - - .. code-block:: bash - - portList: - - label: test - beginAt: 8080 - portNumber: 2 - - Let's say you want to launch a tensorboard in the mnist example using the port. So the first step is to write a wrapper script ``launch_paiYarn.sh`` of ``mnist.py``. - - .. code-block:: bash - - export TENSORBOARD_PORT=paiYarn_PORT_LIST_${paiYarn_CURRENT_TASK_ROLE_NAME}_0_tensorboard - tensorboard --logdir . --port ${!TENSORBOARD_PORT} & - python3 mnist.py - - The config file of portList should be filled as following: - - .. code-block:: yaml - - trial: - command: bash launch_paiYarn.sh - portList: - - label: tensorboard - beginAt: 0 - portNumber: 1 - -NNI support two kind of authorization method in paiYarn, including password and paiYarn token, `refer `__. The authorization is configured in ``paiYarnConfig`` field.\ :raw-html:`
    ` -For password authorization, the ``paiYarnConfig`` schema is: - -.. code-block:: bash - - paiYarnConfig: - userName: your_paiYarn_nni_user - passWord: your_paiYarn_password - host: 10.1.1.1 - -For paiYarn token authorization, the ``paiYarnConfig`` schema is: - -.. code-block:: bash - - paiYarnConfig: - userName: your_paiYarn_nni_user - token: your_paiYarn_token - host: 10.1.1.1 - -Once complete to fill NNI experiment config file and save (for example, save as exp_paiYarn.yml), then run the following command - -.. code-block:: bash - - nnictl create --config exp_paiYarn.yml - -to start the experiment in paiYarn mode. NNI will create OpenpaiYarn job for each trial, and the job name format is something like ``nni_exp_{experiment_id}_trial_{trial_id}``. -You can see jobs created by NNI in the OpenpaiYarn cluster's web portal, like: - -.. image:: ../../img/nni_pai_joblist.jpg - :target: ../../img/nni_pai_joblist.jpg - :alt: - - -Notice: In paiYarn mode, NNIManager will start a rest server and listen on a port which is your NNI WebUI's port plus 1. For example, if your WebUI port is ``8080``\ , the rest server will listen on ``8081``\ , to receive metrics from trial job running in Kubernetes. So you should ``enable 8081`` TCP port in your firewall rule to allow incoming traffic. - -Once a trial job is completed, you can goto NNI WebUI's overview page (like http://localhost:8080/oview) to check trial's information. - -Expand a trial information in trial list view, click the logPath link like: - -.. image:: ../../img/nni_webui_joblist.jpg - :target: ../../img/nni_webui_joblist.jpg - :alt: - - -And you will be redirected to HDFS web portal to browse the output files of that trial in HDFS: - -.. image:: ../../img/nni_trial_hdfs_output.jpg - :target: ../../img/nni_trial_hdfs_output.jpg - :alt: - - -You can see there're three fils in output folder: stderr, stdout, and trial.log - -data management ---------------- - -If your training data is not too large, it could be put into codeDir, and nni will upload the data to hdfs, or you could build your own docker image with the data. If you have large dataset, it's not appropriate to put the data in codeDir, and you could follow the `guidance `__ to mount the data folder in container. - -If you also want to save trial's other output into HDFS, like model files, you can use environment variable ``NNI_OUTPUT_DIR`` in your trial code to save your own output files, and NNI SDK will copy all the files in ``NNI_OUTPUT_DIR`` from trial's container to HDFS, the target path is ``hdfs://host:port/{username}/nni/{experiments}/{experimentId}/trials/{trialId}/nnioutput`` - -version check -------------- - -NNI support version check feature in since version 0.6. It is a policy to insure the version of NNIManager is consistent with trialKeeper, and avoid errors caused by version incompatibility. -Check policy: - - -#. NNIManager before v0.6 could run any version of trialKeeper, trialKeeper support backward compatibility. -#. Since version 0.6, NNIManager version should keep same with triakKeeper version. For example, if NNIManager version is 0.6, trialKeeper version should be 0.6 too. -#. Note that the version check feature only check first two digits of version.For example, NNIManager v0.6.1 could use trialKeeper v0.6 or trialKeeper v0.6.2, but could not use trialKeeper v0.5.1 or trialKeeper v0.7. - -If you could not run your experiment and want to know if it is caused by version check, you could check your webUI, and there will be an error message about version check. - -.. image:: ../../img/version_check.png - :target: ../../img/version_check.png - :alt: - diff --git a/docs/en_US/TrainingService/RemoteMachineMode.rst b/docs/en_US/TrainingService/RemoteMachineMode.rst index c96d5cd660..d2c1660f54 100644 --- a/docs/en_US/TrainingService/RemoteMachineMode.rst +++ b/docs/en_US/TrainingService/RemoteMachineMode.rst @@ -217,3 +217,10 @@ If you want multiple commands to be executed, you can use ``&&`` to connect thes preCommand: command1 && command2 && command3 **Note**\ : Because **preCommand** will execute before other commands each time, it is strongly not recommended to set **preCommand** that will make changes to system, i.e. ``mkdir`` or ``touch``. + +Remote machine support running experiment in reuse mode. In this mode, NNI will reuse remote machine jobs to run as many as possible trials. It can save time of creating new jobs. User needs to make sure each trial can run independent in same job, for example, avoid loading checkpoint from previous trials. +Follow the setting to enable reuse mode: +.. code-block:: yaml + + remoteConfig: + reuse: true \ No newline at end of file diff --git a/docs/en_US/training_services.rst b/docs/en_US/training_services.rst index bb3081fd07..f94db16d16 100644 --- a/docs/en_US/training_services.rst +++ b/docs/en_US/training_services.rst @@ -6,7 +6,6 @@ Introduction to NNI Training Services Local<./TrainingService/LocalMode> Remote<./TrainingService/RemoteMachineMode> OpenPAI<./TrainingService/PaiMode> - OpenPAI Yarn Mode<./TrainingService/PaiYarnMode> Kubeflow<./TrainingService/KubeflowMode> AdaptDL<./TrainingService/AdaptDLMode> FrameworkController<./TrainingService/FrameworkControllerMode> diff --git a/examples/trials/auto-gbdt/config_paiYarn.yml b/examples/trials/auto-gbdt/config_paiYarn.yml deleted file mode 100644 index 427a6eacd8..0000000000 --- a/examples/trials/auto-gbdt/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_auto-gbdt -trialConcurrency: 1 -maxExecDuration: 10h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: minimize -trial: - command: python3 main.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/cifar10_pytorch/config_paiYarn.yml b/examples/trials/cifar10_pytorch/config_paiYarn.yml deleted file mode 100644 index 3ac750f536..0000000000 --- a/examples/trials/cifar10_pytorch/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_pytorch_cifar10 -trialConcurrency: 1 -maxExecDuration: 100h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 main.py - codeDir: . - gpuNum: 1 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 diff --git a/examples/trials/efficientnet/config_paiYarn.yml b/examples/trials/efficientnet/config_paiYarn.yml deleted file mode 100644 index 5c39282211..0000000000 --- a/examples/trials/efficientnet/config_paiYarn.yml +++ /dev/null @@ -1,28 +0,0 @@ -authorName: unknown -experimentName: example_efficient_net -trialConcurrency: 8 -maxExecDuration: 48h -maxTrialNum: 100 -trainingServicePlatform: paiYarn -searchSpacePath: search_net.json -useAnnotation: false -tuner: - codeDir: . - classFileName: tuner.py - className: FixedProductTuner - classArgs: - product: 2 -trial: - codeDir: EfficientNet-PyTorch - command: sh train_imagenet.sh - cpuNum: 4 - memoryMB: 25000 - shmMB: 25000 - gpuNum: 1 - virtualCluster: nni - image: msranni/nni:latest -nniManagerIp: -paiYarnConfig: - userName: - passWord: - host: diff --git a/examples/trials/ga_squad/config_paiYarn.yml b/examples/trials/ga_squad/config_paiYarn.yml deleted file mode 100644 index 4bded4540e..0000000000 --- a/examples/trials/ga_squad/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_ga_squad -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -#choice: true, false -useAnnotation: false -#Your nni_manager ip -nniManagerIp: 10.10.10.10 -tuner: - codeDir: ../../tuners/ga_customer_tuner - classFileName: customer_tuner.py - className: CustomerTuner - classArgs: - optimize_mode: maximize -trial: - command: chmod +x ./download.sh && ./download.sh && python3 trial.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 32869 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 diff --git a/examples/trials/mnist-advisor/config_paiYarn.yml b/examples/trials/mnist-advisor/config_paiYarn.yml deleted file mode 100644 index 2285c23256..0000000000 --- a/examples/trials/mnist-advisor/config_paiYarn.yml +++ /dev/null @@ -1,38 +0,0 @@ -authorName: default -experimentName: example_mnist_hyperband -maxExecDuration: 1h -maxTrialNum: 10000 -trialConcurrency: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -advisor: - #choice: Hyperband, BOHB - #(BOHB should be installed through nnictl) - builtinAdvisorName: Hyperband - classArgs: - #R: the maximum trial budget - R: 100 - #eta: proportion of discarded trials - eta: 3 - #choice: maximize, minimize - optimize_mode: maximize - #choice: serial, parallelism - exec_mode: parallelism -trial: - command: python3 mnist.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 diff --git a/examples/trials/mnist-annotation/config_paiYarn.yml b/examples/trials/mnist-annotation/config_paiYarn.yml deleted file mode 100644 index 1a3299d606..0000000000 --- a/examples/trials/mnist-annotation/config_paiYarn.yml +++ /dev/null @@ -1,31 +0,0 @@ -authorName: default -experimentName: example_mnist -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -#choice: true, false -useAnnotation: true -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 mnist.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/mnist-batch-tune-keras/config_paiYarn.yml b/examples/trials/mnist-batch-tune-keras/config_paiYarn.yml deleted file mode 100644 index a81932285f..0000000000 --- a/examples/trials/mnist-batch-tune-keras/config_paiYarn.yml +++ /dev/null @@ -1,29 +0,0 @@ -authorName: default -experimentName: example_mnist-keras -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: BatchTuner -trial: - command: python3 mnist-keras.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 diff --git a/examples/trials/mnist-keras/config_paiYarn.yml b/examples/trials/mnist-keras/config_paiYarn.yml deleted file mode 100644 index 4e5279a689..0000000000 --- a/examples/trials/mnist-keras/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_mnist-keras -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 mnist-keras.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/mnist-pytorch/config_paiYarn.yml b/examples/trials/mnist-pytorch/config_paiYarn.yml deleted file mode 100644 index d1aae75122..0000000000 --- a/examples/trials/mnist-pytorch/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_mnist_pytorch -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 mnist.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/mnist-tfv1/config_paiYarn.yml b/examples/trials/mnist-tfv1/config_paiYarn.yml deleted file mode 100644 index 886ee21c09..0000000000 --- a/examples/trials/mnist-tfv1/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_mnist -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 mnist.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml b/examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml deleted file mode 100644 index e15ec2ebec..0000000000 --- a/examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml +++ /dev/null @@ -1,39 +0,0 @@ -authorName: default -experimentName: example_FashionMNIST-network-morphism -trialConcurrency: 1 -maxExecDuration: 24h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, NetworkMorphism - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: NetworkMorphism - classArgs: - #choice: maximize, minimize - optimize_mode: maximize - # for now, this tuner only supports cv domain - task: cv - #input image width - input_width: 28 - #input image channel - input_channel: 1 - #number of classes - n_output_node: 10 -trial: - command: python3 FashionMNIST_keras.py - codeDir: . - gpuNum: 1 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/network_morphism/cifar10/config_paiYarn.yml b/examples/trials/network_morphism/cifar10/config_paiYarn.yml deleted file mode 100644 index 3367aa4e36..0000000000 --- a/examples/trials/network_morphism/cifar10/config_paiYarn.yml +++ /dev/null @@ -1,39 +0,0 @@ -authorName: default -experimentName: example_cifar10-network-morphism -trialConcurrency: 1 -maxExecDuration: 24h -maxTrialNum: 10 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, NetworkMorphism - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: NetworkMorphism - classArgs: - #choice: maximize, minimize - optimize_mode: maximize - # for now, this tuner only supports cv domain - task: cv - #input image width - input_width: 32 - #input image channel - input_channel: 3 - #number of classes - n_output_node: 10 -trial: - command: python3 cifar10_keras.py - codeDir: . - gpuNum: 1 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/sklearn/classification/config_paiYarn.yml b/examples/trials/sklearn/classification/config_paiYarn.yml deleted file mode 100644 index 9bec9a4c50..0000000000 --- a/examples/trials/sklearn/classification/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_sklearn -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 100 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner,MetisTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 main.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file diff --git a/examples/trials/sklearn/regression/config_paiYarn.yml b/examples/trials/sklearn/regression/config_paiYarn.yml deleted file mode 100644 index 0e0e73a0ab..0000000000 --- a/examples/trials/sklearn/regression/config_paiYarn.yml +++ /dev/null @@ -1,32 +0,0 @@ -authorName: default -experimentName: example_sklearn -trialConcurrency: 1 -maxExecDuration: 1h -maxTrialNum: 100 -#choice: local, remote, pai -trainingServicePlatform: paiYarn -searchSpacePath: search_space.json -#choice: true, false -useAnnotation: false -tuner: - #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner - #SMAC (SMAC should be installed through nnictl) - builtinTunerName: TPE - classArgs: - #choice: maximize, minimize - optimize_mode: maximize -trial: - command: python3 main.py - codeDir: . - gpuNum: 0 - cpuNum: 1 - memoryMB: 8196 - #The docker image to run nni job on pai - image: msranni/nni:latest -paiYarnConfig: - #The username to login pai - userName: username - #The password to login pai - passWord: password - #The host of restful server of pai - host: 10.10.10.10 \ No newline at end of file From 2c4934af143379db2dad260fa34db7e8bce8216e Mon Sep 17 00:00:00 2001 From: Shinai Yang Date: Mon, 4 Jan 2021 12:08:57 +0800 Subject: [PATCH 3/5] format doc --- docs/en_US/TrainingService/RemoteMachineMode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en_US/TrainingService/RemoteMachineMode.rst b/docs/en_US/TrainingService/RemoteMachineMode.rst index d2c1660f54..d697dd4c71 100644 --- a/docs/en_US/TrainingService/RemoteMachineMode.rst +++ b/docs/en_US/TrainingService/RemoteMachineMode.rst @@ -218,7 +218,7 @@ If you want multiple commands to be executed, you can use ``&&`` to connect thes **Note**\ : Because **preCommand** will execute before other commands each time, it is strongly not recommended to set **preCommand** that will make changes to system, i.e. ``mkdir`` or ``touch``. -Remote machine support running experiment in reuse mode. In this mode, NNI will reuse remote machine jobs to run as many as possible trials. It can save time of creating new jobs. User needs to make sure each trial can run independent in same job, for example, avoid loading checkpoint from previous trials. +Remote machine support running experiment in reuse mode. In this mode, NNI will reuse remote machine jobs to run as many as possible trials. It can save time of creating new jobs. User needs to make sure each trial can run independent in same job, for example, avoid loading checkpoint from previous trials. Follow the setting to enable reuse mode: .. code-block:: yaml From fd78d9aef80286790ceda721b0cebe8bcdb7905c Mon Sep 17 00:00:00 2001 From: Shinai Yang Date: Mon, 4 Jan 2021 12:09:51 +0800 Subject: [PATCH 4/5] format doc --- docs/en_US/TrainingService/RemoteMachineMode.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/en_US/TrainingService/RemoteMachineMode.rst b/docs/en_US/TrainingService/RemoteMachineMode.rst index d697dd4c71..83a90a3aeb 100644 --- a/docs/en_US/TrainingService/RemoteMachineMode.rst +++ b/docs/en_US/TrainingService/RemoteMachineMode.rst @@ -220,6 +220,7 @@ If you want multiple commands to be executed, you can use ``&&`` to connect thes Remote machine support running experiment in reuse mode. In this mode, NNI will reuse remote machine jobs to run as many as possible trials. It can save time of creating new jobs. User needs to make sure each trial can run independent in same job, for example, avoid loading checkpoint from previous trials. Follow the setting to enable reuse mode: + .. code-block:: yaml remoteConfig: From 5b9f41450cf95f930f1421fe9efa831da8c052a4 Mon Sep 17 00:00:00 2001 From: Shinai Yang Date: Tue, 5 Jan 2021 10:37:59 +0800 Subject: [PATCH 5/5] fix comments --- docs/en_US/TrainingService/RemoteMachineMode.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en_US/TrainingService/RemoteMachineMode.rst b/docs/en_US/TrainingService/RemoteMachineMode.rst index 83a90a3aeb..fba51c5abb 100644 --- a/docs/en_US/TrainingService/RemoteMachineMode.rst +++ b/docs/en_US/TrainingService/RemoteMachineMode.rst @@ -218,7 +218,7 @@ If you want multiple commands to be executed, you can use ``&&`` to connect thes **Note**\ : Because **preCommand** will execute before other commands each time, it is strongly not recommended to set **preCommand** that will make changes to system, i.e. ``mkdir`` or ``touch``. -Remote machine support running experiment in reuse mode. In this mode, NNI will reuse remote machine jobs to run as many as possible trials. It can save time of creating new jobs. User needs to make sure each trial can run independent in same job, for example, avoid loading checkpoint from previous trials. +Remote machine supports running experiment in reuse mode. In this mode, NNI will reuse remote machine jobs to run as many as possible trials. It can save time of creating new jobs. User needs to make sure each trial can run independent in the same job, for example, avoid loading checkpoint from previous trials. Follow the setting to enable reuse mode: .. code-block:: yaml
    drawing