Skip to content

Commit

Permalink
dummy example
Browse files Browse the repository at this point in the history
  • Loading branch information
zheyuye committed Aug 18, 2020
1 parent 633e683 commit 859ab4d
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 4 deletions.
16 changes: 14 additions & 2 deletions tools/batch/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
# Batch Job Examples
Based on AWS Batch, we provide a quick way to submit a job, including Conversion, Question Answering, Machine Translation
Text Translation and Pre-trained Model Training, which can be used for regular inspections or for rapid deployment of experiments.
Based on AWS Batch, we provide a quick way to submit a job, including Conversion, Question Answering, Machine Translation, Text Translation and Pre-trained Model Training, which can be used for regular inspections or for rapid deployment of experiments.

See a dummy submission for example in which `remote` and `source-ref` denote the repository url and branch name respectively.

```bash
python3 submit-job.py \
--region us-east-1 \
--source-ref master \
--job-type g4dn.12x \
--name dummy_example \
--work-dir scripts/preprocess \
--remote https://github.com/dmlc/gluon-nlp/ \
--command 'git rev-parse HEAD | tee stdout.log'
```
## Conversion Toolkits
Following the instruction of [converting scripts](../../scripts/conversion_toolkits), several pre-trained models could be converted through the corresponding conversion tool as below command where `${MODEL_TYPE}` could be selected from `[albert, bert, electra, mobilebert, bart, robert, xmlr]`.
```bash
Expand Down
4 changes: 2 additions & 2 deletions tools/batch/submit-job.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
'p3.2x', 'p3.8x', 'p3.16x', 'p3dn.24x',
'c5n.18x'], default='g4dn.4x')
parser.add_argument('--source-ref',
help='ref in GluonNLP main github. e.g. numpy, refs/pull/500/head',
type=str, default='numpy')
help='ref in GluonNLP main github. e.g. master, refs/pull/500/head',
type=str, default='master')
parser.add_argument('--work-dir',
help='working directory inside the repo. e.g. scripts/preprocess',
type=str, default='scripts/preprocess')
Expand Down

0 comments on commit 859ab4d

Please sign in to comment.