-
Notifications
You must be signed in to change notification settings - Fork 538
[CI] Update GPU Test Workflow + Update Some Tests and README #1316
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1316 +/- ##
=======================================
Coverage 84.45% 84.45%
=======================================
Files 42 42
Lines 6426 6426
=======================================
Hits 5427 5427
Misses 999 999
Continue to review full report at Codecov.
|
.github/workflows/unittests-gpu.yml
Outdated
@@ -38,9 +38,6 @@ jobs: | |||
|
|||
- name: Test project on AWS Batch | |||
run: | | |||
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "./test.sh" --wait | |||
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "../../test.sh" --wait | tee > script.log | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to manually upload the coverage in AWS Batch?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also use g4.2x
cfg.MODEL.num_heads = 2 | ||
cfg.MODEL.compute_layout = compute_layout | ||
cfg.freeze() | ||
def test_bert_small_cfg(compute_layout, ctx): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. We can use this like mx.context._current.set(mx.Context('gpu',0))
without with
statement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an internal API and may change in the future. It may be better to use the with
statement inside the conftest.py
test.sh
Outdated
python3 -m pip install --upgrade --quiet cython | ||
python3 -m pip install --pre --user --quiet "mxnet-cu102>=2.0.0b20200802" -f https://dist.mxnet.io/python | ||
python3 -m pip install --user --quiet -e .[extras] | ||
python3 -m pytest --cov=./ --cov-report=xml --durations=50 --device="gpu" ../../tests/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this script is already at the root level and looking in paths further up would be confusing.
test.sh
Outdated
@@ -3,10 +3,12 @@ | |||
|
|||
# alias python3='/usr/bin/python3' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why comment?
.github/workflows/unittests-gpu.yml
Outdated
@@ -38,9 +38,6 @@ jobs: | |||
|
|||
- name: Test project on AWS Batch | |||
run: | | |||
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "./test.sh" --wait | |||
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "../../test.sh" --wait | tee > script.log | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also use g4.2x
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Description
Checklist
Essentials
Changes
Comments
cc @dmlc/gluon-nlp-team