Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Add MXNet Ops for fast multihead attention #16408

Merged
merged 21 commits into from
Nov 7, 2019
Merged

Add MXNet Ops for fast multihead attention #16408

merged 21 commits into from
Nov 7, 2019

Commits on Oct 9, 2019

  1. add MXNet Ops for fast multihead attention

    @Caenorst
    Caenorst committed Oct 9, 2019
    Configuration menu
    Copy the full SHA
    e987614 View commit details
    Browse the repository at this point in the history

  2. add cutlass as 3rdparty dependency

    @Caenorst
    Caenorst committed Oct 9, 2019
    Configuration menu
    Copy the full SHA
    cbbeac9 View commit details
    Browse the repository at this point in the history

  3. add cutlass to compilation flags

    @Caenorst
    Caenorst committed Oct 9, 2019
    Configuration menu
    Copy the full SHA
    7a46c6a View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2019

  1. remove all cutlass stuff

    @Caenorst
    Caenorst committed Oct 15, 2019
    Configuration menu
    Copy the full SHA
    7fe71f9 View commit details
    Browse the repository at this point in the history

  2. add better error message and description and remove cutlass from comp… 

    …ilation flags
    @Caenorst
    Caenorst committed Oct 15, 2019
    Configuration menu
    Copy the full SHA
    fe8bb48 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2019

  1. change credit for the approach since the code have changed

    @Caenorst
    Caenorst committed Oct 16, 2019
    Configuration menu
    Copy the full SHA
    b8505cd View commit details
    Browse the repository at this point in the history

  2. fix typos

    @Caenorst
    Caenorst committed Oct 16, 2019
    Configuration menu
    Copy the full SHA
    eb2e708 View commit details
    Browse the repository at this point in the history

  3. correct another typo

    @Caenorst
    Caenorst committed Oct 16, 2019
    Configuration menu
    Copy the full SHA
    6c18f13 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2019

  1. Add all the cuda/cublas helper functions

    @Caenorst
    Caenorst committed Oct 18, 2019
    Configuration menu
    Copy the full SHA
    abe31ac View commit details
    Browse the repository at this point in the history

  2. remove tests using kAddTo

    @Caenorst
    Caenorst committed Oct 18, 2019
    Configuration menu
    Copy the full SHA
    2e174e9 View commit details
    Browse the repository at this point in the history

  3. only use cublasStridedBatchedGemm if CUDA >= 9.1

    @Caenorst
    Caenorst committed Oct 18, 2019
    Configuration menu
    Copy the full SHA
    fc77243 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2019

  1. add equivalent mxnet code in description of mha ops

    @Caenorst
    Caenorst committed Oct 21, 2019
    Configuration menu
    Copy the full SHA
    b0823c3 View commit details
    Browse the repository at this point in the history

  2. remove a wrong copy-paste

    @Caenorst
    Caenorst committed Oct 21, 2019
    Configuration menu
    Copy the full SHA
    95bee18 View commit details
    Browse the repository at this point in the history

  3. add _contrib for namespace and add GPU only on description

    @Caenorst
    Caenorst committed Oct 21, 2019
    Configuration menu
    Copy the full SHA
    790701d View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2019

  1. add warning in bwd_ignore_zero_init description, also test with fp32

    @Caenorst
    Caenorst committed Oct 31, 2019
    Configuration menu
    Copy the full SHA
    6004ac9 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2019

  1. add error return if bwd_ignore_zero_init is used without MXNET_EXEC_E… 

    …NABLE_ADDTO
    @Caenorst
    Caenorst committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    ff36026 View commit details
    Browse the repository at this point in the history

  2. remove std::move for clang

    @Caenorst
    Caenorst committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    f3eee60 View commit details
    Browse the repository at this point in the history

  3. remove bwd_ignore_zero_init flag

    @Caenorst
    Caenorst committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    d430ca6 View commit details
    Browse the repository at this point in the history

  4. remove bwd_ignore_zero_init in test_operator_gpu.py

    @Caenorst
    Caenorst committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    1a2b35c View commit details
    Browse the repository at this point in the history

  5. fix typo

    @Caenorst
    Caenorst committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    3e408c8 View commit details
    Browse the repository at this point in the history

  6. fix another typo

    @Caenorst
    Caenorst committed Nov 1, 2019
    Configuration menu
    Copy the full SHA
    dbcb5b5 View commit details
    Browse the repository at this point in the history