Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[FEATURE] Add interleaved batch_dot oneDNN fuses for new GluonNLP models #20312

Merged
merged 8 commits into from
Jun 29, 2021
Merged

[FEATURE] Add interleaved batch_dot oneDNN fuses for new GluonNLP models #20312

merged 8 commits into from
Jun 29, 2021

Conversation

bgawrych
Copy link
Contributor

Description

This change utilizes oneDNN matmul primitive capabilities and fuses following sequences of operators:
image

Tested on run_squad.py script from GluonNLP repository (weights downloaded from QA README.md file)
image

  • electra-small quantized with disabled 1st fully connected layer (there is bug not related to this change) - fix incoming
  • accuracy drop on bert base is caused because there wasn't custom calibration - without this change quantized accuracy is similar (69.0617)

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage

@bgawrych bgawrych requested a review from szha as a code owner May 26, 2021 13:39
@mxnet-bot
Copy link

Hey @bgawrych , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-gpu, clang, unix-gpu, edge, centos-gpu, website, sanity, unix-cpu, centos-cpu, miscellaneous, windows-cpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels May 26, 2021
@bgawrych
Copy link
Contributor Author

bgawrych commented Jun 1, 2021

Hi @leezu @szha, is something wrong with master CI? I get always

[2021-06-01T09:46:09.673Z] =========================== short test summary info ===========================

[2021-06-01T09:46:09.673Z] FAILED tests/python/gpu/test_operator_gpu.py::test_sparse_nd_pickle - mxnet.b...

[2021-06-01T09:46:09.673Z] FAILED tests/python/gpu/test_operator_gpu.py::test_sparse_nd_save_load[save]

[2021-06-01T09:46:09.673Z] FAILED tests/python/gpu/test_operator_gpu.py::test_sparse_ndarray_load_csr_npz_scipy[save]

on windows-gpu - in this PR and in #20227

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jun 2, 2021
@szha
Copy link
Member

szha commented Jun 2, 2021

If the tests fail consistently, let's disable these tests and open an issue for tracking them.

@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jun 2, 2021
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jun 9, 2021
@bgawrych
Copy link
Contributor Author

@akarbown Can you review and merge if it's ok?

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jun 29, 2021
@bgawrych
Copy link
Contributor Author

@mxnet-bot run ci [unix-cpu, windows-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu, windows-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jun 29, 2021
@akarbown akarbown merged commit 1d0bdfd into apache:master Jun 29, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@bartekkuncer bartekkuncer bartekkuncer left review comments

@szha szha Awaiting requested review from szha szha is a code owner

Assignees
No one assigned
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@bgawrych @mxnet-bot @szha @bartekkuncer @mseth10 @akarbown