Skip to content

Conversation

@samnordmann
Copy link
Collaborator

@samnordmann samnordmann commented Apr 27, 2022

What

Add support for four new datatypes: float128, float32_complex, float64_complex, float128_complex.

  • float32_complex and float64_complex are supported on HOST and CUDA. float128 and float128_complex are supported only on HOST.
  • The new datatypes are supported by all collectives. float128 supported with the reduction ops: min, max, sum, prod, avg. float32(64,128)_complex supported with the reduction ops: sum, prod, avg.
  • Update MPI_test and gtest with the new datatypes. Remark: the gtests are run with float128 and float128 only on mc_reduce tests.
  • new datatypes not supported by nccl nor sharp TLs

@samnordmann samnordmann changed the title add support for float128 datype on CPU, all collectives, and reductio… API: add float128 and float32(64,128)_complex dt Apr 27, 2022
@samnordmann samnordmann force-pushed the complex_dt branch 2 times, most recently from 128cba3 to 80f7bd6 Compare April 27, 2022 11:45
@manjugv
Copy link
Contributor

manjugv commented Apr 27, 2022

@samnordmann It might be better to decouple the formatting changes from the functional changes.

Copy link

@jladd-mlnx jladd-mlnx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Way too many white-space changes. I had to hide whitespaces in order to make sense of the PR.

  • The file src/ucc/api/ucc.h is Whitespace-only changes.

  • Should we include support for complex half precision? Does HPL AI use it?

typedef enum cudaDataType_t
{
        CUDA_R_16F= 2, // 16 bit real 
        CUDA_C_16F= 6, // 16 bit complex
        CUDA_R_32F= 0, // 32 bit real
        CUDA_C_32F= 4, // 32 bit complex
        CUDA_R_64F= 1, // 64 bit real
        CUDA_C_64F= 5, // 64 bit complex
        CUDA_R_8I= 3,  // 8 bit real as a signed integer 
        CUDA_C_8I= 7,  // 8 bit complex as a pair of signed integers
        CUDA_R_8U= 8,  // 8 bit real as a signed integer 
        CUDA_C_8U= 9   // 8 bit complex as a pair of signed integers
} cudaDataType;

@vspetrov vspetrov mentioned this pull request May 5, 2022
@samnordmann samnordmann force-pushed the complex_dt branch 3 times, most recently from 8b42a71 to 20d60a8 Compare May 9, 2022 09:24
@samnordmann
Copy link
Collaborator Author

samnordmann commented May 9, 2022

@samnordmann It might be better to decouple the formatting changes from the functional changes.

  • Way too many white-space changes. I had to hide whitespaces in order to make sense of the PR.
  • The file src/ucc/api/ucc.h is Whitespace-only changes.

The formatting changes were due to a mistake with automatic formatting. It is fixed now.
@jladd-mlnx @manjugv

@samnordmann samnordmann force-pushed the complex_dt branch 2 times, most recently from cd992f2 to 9bed721 Compare May 9, 2022 09:44
@samnordmann samnordmann requested a review from shimmybalsam May 9, 2022 11:16
@samnordmann samnordmann force-pushed the complex_dt branch 5 times, most recently from db2ae78 to 3a5f501 Compare May 12, 2022 12:27
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not related to this PR, but there are SHARP_DTYPE_UINT8 and SHARP_DTYPE_UINT8 defined in sharp.h, should we update mapping? cc @bureddy

@samnordmann samnordmann force-pushed the complex_dt branch 2 times, most recently from 6ee4591 to 4b54615 Compare June 17, 2022 23:29
@vspetrov
Copy link
Collaborator

bot:retest

@samnordmann samnordmann force-pushed the complex_dt branch 5 times, most recently from 2cb6da8 to 50ae438 Compare June 20, 2022 13:16
Copy link
Collaborator

@vspetrov vspetrov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samnordmann plz address final set of minor changes

@vspetrov vspetrov merged commit 8afd34a into openucx:master Jun 21, 2022
Sergei-Lebedev pushed a commit to Sergei-Lebedev/ucc that referenced this pull request Jun 22, 2022
* API: add float128 and float32(64,128)_complex dt

* TEST: update mpi_tests with new dt

* TEST: update Gtest with new dt

* BUILD: check dt size during preprocessing
kingchc pushed a commit to facebookresearch/ucc that referenced this pull request Jul 20, 2022
* API: add float128 and float32(64,128)_complex dt

* TEST: update mpi_tests with new dt

* TEST: update Gtest with new dt

* BUILD: check dt size during preprocessing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants