Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[CI] Update GPU Test Workflow + Update Some Tests and README #1316

Merged
merged 39 commits into from
Aug 28, 2020

Conversation

barry-jin
Copy link
Contributor

Description

  1. Make gpu test workflow sensitive to pytest results.
  2. Add GPU tests for Attention Cells, Bert, Electra.
  3. Update README for tests.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

barry-jin and others added 30 commits August 21, 2020 13:52
[CI] Update unittests-gpu (dmlc#1313)
automatically generate date suffix for dev versions (dmlc#1314)
@barry-jin barry-jin requested a review from a team as a code owner August 26, 2020 04:10
@codecov
Copy link

codecov bot commented Aug 26, 2020

Codecov Report

Merging #1316 into master will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1316   +/-   ##
=======================================
  Coverage   84.45%   84.45%           
=======================================
  Files          42       42           
  Lines        6426     6426           
=======================================
  Hits         5427     5427           
  Misses        999      999           
Impacted Files Coverage Δ
conftest.py 85.29% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 970318d...9d4c459. Read the comment docs.

@@ -38,9 +38,6 @@ jobs:

- name: Test project on AWS Batch
run: |
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "./test.sh" --wait
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "../../test.sh" --wait | tee > script.log

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to manually upload the coverage in AWS Batch?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also use g4.2x

cfg.MODEL.num_heads = 2
cfg.MODEL.compute_layout = compute_layout
cfg.freeze()
def test_bert_small_cfg(compute_layout, ctx):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We can use this like mx.context._current.set(mx.Context('gpu',0)) without with statement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an internal API and may change in the future. It may be better to use the with statement inside the conftest.py

test.sh Outdated
python3 -m pip install --upgrade --quiet cython
python3 -m pip install --pre --user --quiet "mxnet-cu102>=2.0.0b20200802" -f https://dist.mxnet.io/python
python3 -m pip install --user --quiet -e .[extras]
python3 -m pytest --cov=./ --cov-report=xml --durations=50 --device="gpu" ../../tests/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this script is already at the root level and looking in paths further up would be confusing.

test.sh Outdated
@@ -3,10 +3,12 @@

# alias python3='/usr/bin/python3'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why comment?

@@ -38,9 +38,6 @@ jobs:

- name: Test project on AWS Batch
run: |
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "./test.sh" --wait
python ./tools/batch/submit-job.py --region us-east-1 --job-type p3.2x --source-ref ${{ github.ref }} --work-dir tools/batch --remote https://github.com/dmlc/gluon-nlp --command "../../test.sh" --wait | tee > script.log

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also use g4.2x

Copy link
Member

@sxjscience sxjscience left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sxjscience sxjscience merged commit bba8697 into dmlc:master Aug 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants