mpi: embiggening #4961

hzhou · 2020-12-17T04:12:42Z

Pull Request Description

This is a work-in-progress to implement the MPI 4.0 large count.

[skip warnings]

Expected Impact

Author Checklist

hzhou · 2020-12-17T04:13:25Z

Special warning test to warn about integer conversion (-Wconversion)

test:mpich/warnings/int

hzhou · 2020-12-17T05:57:08Z

test:mpich/warnings/int

hzhou · 2020-12-17T17:22:45Z

test:mpich/warnings/int

src/mpid/ch4/netmod/ofi/ofi_am_impl.h

raffenet

Looks OK to me.

maint/gen_binding_c.py

hzhou · 2021-01-13T19:32:20Z

Looks OK to me.

Thanks. I'll split some of the warnings change for you to review and approve again. The embiggening is not done yet.

significant changes

hzhou · 2021-02-18T05:00:51Z

test:mpich/ch3/most
test:mpich/ch4/most

raffenet

I didn't see much to object to here. I'm a bit confused by the commit I commented on, both otherwise seems OK.

raffenet · 2021-02-19T18:39:25Z

src/include/mpir_datatype.h

 void MPIR_Type_free_impl(MPI_Datatype * datatype);
-void MPIR_Pack_size_impl(int incount, MPI_Datatype datatype, MPI_Aint * size);
+void MPIR_Pack_size(MPI_Aint incount, MPI_Datatype datatype, MPI_Aint * size);


I'm not sure exactly what's happening in this commit. It seems datatype _impl functions are becoming wrappers where they weren't before.
For example:
(old) MPI_Pack_size -> MPIR_Pack_size_impl
(new) MPI_Pack_size -> MPIR_Pack_size -> MPIR_Pack_size_impl
Is this so small and large (_c) versions can be treated differently before calling the _impl API?

I'm not sure exactly what's happening in this commit. It seems datatype _impl functions are becoming wrappers where they weren't before.
For example:
(old) MPI_Pack_size -> MPIR_Pack_size_impl
(new) MPI_Pack_size -> MPIR_Pack_size -> MPIR_Pack_size_impl
Is this so small and large (_c) versions can be treated differently before calling the _impl API?

In the large branch we may swap the array or may not, depending on whether MPI_Aint is the same size as MPI_Count. When we do the swap, the parameter has to be renamed. This becomes awkward for inlined "body of routines" since to have two large_count versions with different variable names seem just silly. Thus it seems much easier to just move the inline code block into _impl wrapper, which can be handled automatically.

I see. Can you add this more detailed description to the commit message? At the moment

datatype: use MPI_Aint for internal impl functions

lacks information about the addition of the wrappers.

hzhou · 2021-02-19T21:34:50Z

test:mpich/ch4/ofi

raffenet

Just one minor nitpick.

raffenet · 2021-02-19T21:57:00Z

src/mpi/datatype/datatype_impl.c

+    if (mpi_errno)
+        goto fn_fail;


nitpick: should this be MPI_ERR_CHECK(mpi_errno)?

Oh, I see this was just copy/paste from before. No need to fix now.

Thanks. I'll fix it before merging.

Need use MPI_Aint internally for buffer sizes to accommodate large count.

This function must be a leftover from development. It has never been used and I don't think it is complete anyway.

Need differentiate internal func attributes from the user configured attributes.

While it is technically safe, having the same temparary name within nested scope is confusing. Rename to clarity.

This will be used when user use a small function and the value returned won't fit in the output parameter.

We will call dump_mpi_c twice for functions with POLY types. First we call with map_type="SMALL", then call with map_type="BIG". Functions without POLY types will just call dump_mpi_c once with map_type="SMALL". First of all, if there is a custom "large_count" block, we'll always use that in large version. If a POLY function only has scalar POLY type, we assume the implementation use MPI_Aint -- need fix any exceptions. If a POLY function has scalar output or inout POLY type, we assume internally use MPI_Aint, and python generrates temporary MPI_Aint variable to copy in and out value in the *small* version. If the function uses POLY arrays, we will use separate internal routines for datatype functions and always use MPI_Aint for collective (and other if any) functions. The array swap can be messy due to function-specific array lengths. We took the way out by only doing array swap in v-collectives. large mpir impl version with `_c_impl` suffix, and will call that. The compilation will fail if that (_c_impl) routine is missing. When we only use MPI_Aint for internal impl functions, we also check and convert when MPI_Count is bigger than MPI_Aint (such as the case in 32-bit systems).

The parameter mapping is passed along only because a few lower level function need it. Since the mapping concept is obscure at higher level, having it in the interface makes the code obscrue. On the other hand, a simpl string parameter "map_type" is more intuitive to understand. This commit uses "map_type" or simply omits mapping whereever the mapping is not directly needed.

... that has output counters, e.g. MPI_Get_count, MPI_Get_elements, MPI_Pack, MPI_Unpack, etc. We need swap pointers when the type `MPI_Count` and `MPI_Aint` are not the same size, or we may skip the swap. The swapping will require use of different variables. This become difficult to handle with inlined body-of-routines. Wrap them inside an _impl function allows both cases to be handled properly.

Unfortunately our fortran bindings uses _c_impl and _c to refer to C versions of Fortran binding, causing conflicts with the large count interface. Since it is easier to change the C bindings, we use _large_impl to differentiate. I think it is more readable than _c suffix anyway.

hzhou · 2021-02-20T01:05:58Z

test:mpich/ch3/most
test:mpich/ch4/most

hzhou · 2021-05-29T19:27:43Z

Reference #4880

hzhou force-pushed the 2012_embiggening branch 6 times, most recently from 7356a27 to 0134c73 Compare December 17, 2020 05:56

hzhou force-pushed the 2012_embiggening branch 10 times, most recently from 2f8c1fa to 664cb1b Compare December 26, 2020 02:42

raffenet reviewed Jan 13, 2021

View reviewed changes

src/mpid/ch4/netmod/ofi/ofi_am_impl.h Outdated Show resolved Hide resolved

raffenet previously approved these changes Jan 13, 2021

View reviewed changes

maint/gen_binding_c.py Outdated Show resolved Hide resolved

hzhou force-pushed the 2012_embiggening branch from 664cb1b to 541bb92 Compare January 15, 2021 23:59

hzhou force-pushed the 2012_embiggening branch 4 times, most recently from c6aabbc to 41ea836 Compare February 17, 2021 03:03

hzhou marked this pull request as ready for review February 17, 2021 03:05

hzhou force-pushed the 2012_embiggening branch from 5898fa4 to f9121e3 Compare February 17, 2021 04:33

hzhou force-pushed the 2012_embiggening branch 6 times, most recently from de2d464 to e2bddb9 Compare February 18, 2021 00:38

hzhou force-pushed the 2012_embiggening branch from e2bddb9 to d1a3b03 Compare February 18, 2021 04:59

raffenet reviewed Feb 19, 2021

View reviewed changes

hzhou force-pushed the 2012_embiggening branch from d1a3b03 to 75146fd Compare February 19, 2021 21:34

hzhou requested a review from raffenet February 19, 2021 21:35

raffenet approved these changes Feb 19, 2021

View reviewed changes

hzhou added 11 commits February 19, 2021 19:05

pt2pt: make bsend buffer functions large compatible

1e739c2

Need use MPI_Aint internally for buffer sizes to accommodate large count.

op: add MPIR_Op_create_c_impl

068607a

python: remove unused dump_function_polymorph

d27a0e7

This function must be a leftover from development. It has never been used and I don't think it is complete anyway.

python: add underscore to internal attribute names

ecb945d

Need differentiate internal func attributes from the user configured attributes.

python: use lw_req in early_return routine to avoid confusion

2f07905

While it is technically safe, having the same temparary name within nested scope is confusing. Rename to clarity.

errhan: add errname **too_big_for_output

12edbc0

This will be used when user use a small function and the value returned won't fit in the output parameter.

test: add basic large API tests

deb6de4

hzhou force-pushed the 2012_embiggening branch from 75146fd to deb6de4 Compare February 20, 2021 01:05

hzhou merged commit c23e573 into pmodels:main Feb 20, 2021

hzhou deleted the 2012_embiggening branch February 20, 2021 05:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mpi: embiggening #4961

mpi: embiggening #4961

hzhou commented Dec 17, 2020 •

edited

Loading

hzhou commented Dec 17, 2020 •

edited

Loading

hzhou commented Dec 17, 2020

hzhou commented Dec 17, 2020

raffenet left a comment

hzhou commented Jan 13, 2021

hzhou commented Feb 18, 2021

raffenet left a comment

raffenet Feb 19, 2021

hzhou Feb 19, 2021

raffenet Feb 19, 2021

hzhou Feb 19, 2021

hzhou commented Feb 19, 2021

raffenet left a comment

raffenet Feb 19, 2021

raffenet Feb 19, 2021

hzhou Feb 19, 2021

hzhou commented Feb 20, 2021

hzhou commented May 29, 2021

mpi: embiggening #4961

mpi: embiggening #4961

Conversation

hzhou commented Dec 17, 2020 • edited Loading

Pull Request Description

Expected Impact

Author Checklist

hzhou commented Dec 17, 2020 • edited Loading

hzhou commented Dec 17, 2020

hzhou commented Dec 17, 2020

raffenet left a comment

Choose a reason for hiding this comment

hzhou commented Jan 13, 2021

hzhou commented Feb 18, 2021

raffenet left a comment

Choose a reason for hiding this comment

raffenet Feb 19, 2021

Choose a reason for hiding this comment

hzhou Feb 19, 2021

Choose a reason for hiding this comment

raffenet Feb 19, 2021

Choose a reason for hiding this comment

hzhou Feb 19, 2021

Choose a reason for hiding this comment

hzhou commented Feb 19, 2021

raffenet left a comment

Choose a reason for hiding this comment

raffenet Feb 19, 2021

Choose a reason for hiding this comment

raffenet Feb 19, 2021

Choose a reason for hiding this comment

hzhou Feb 19, 2021

Choose a reason for hiding this comment

hzhou commented Feb 20, 2021

hzhou commented May 29, 2021

hzhou commented Dec 17, 2020 •

edited

Loading

hzhou commented Dec 17, 2020 •

edited

Loading