"WITH_INFO" review correlated for collective operations in Chapters "Collective Operations" and "Process Topologies"

[tonyskjellum]

Problem

Info keys are missing from blocking and nonblocking collective operations.

Blocking and nonblocking APIs (noted below) presently lack info arguments for legacy reasons; per-operation info arguments can be useful to application performance tuning in MPI implementations that support such optional arguments. Standardized info key may also be proposed separately.

If Ticket #80 is approved, this is a proposed extension to those extended APIs to be proposed for collective operations. We do not intend to create a combinatorial explosion of APIs WITH_INFO and _X, but rather merge the two changes into one new API set (_X).

Proposal

The blocking and nonblocking collectives APIs that will be modified by Ticket #80 that do not have info arguments will be augmented to have info arguments. For each case, we will make sure it makes sense for the info argument to be added in the _X interface based on performance and functionality. In some cases, we might prefer to have a separate _WITH_INFO form.

In general, the info argument will be added between the comm and request arguments for nonblocking arguments, and after the comm in blocking operations.

Example (shown with C bindings to emphasize the Ticket #80 changes too):
Present:

Current:
MPI_Ibcast(void* buffer, int count, MPI _Datatype datatype, int root, MPI _Comm comm, MPI_Request *request)

Ticket #85 addition (on its own):
MPI_Ibcast_with_info(void* buffer, int count, MPI _Datatype datatype, int root, MPI _Comm comm, MPI_Info info, MPI_Request *request)

Ticket #80 addition (on its own):
MPI_Ibcast_x(void* buffer, MPI_Count count, MPI _Datatype datatype, int root, MPI _Comm comm, MPI_Request *request)

Both tickets #80 and #85 are accepted together:
MPI_Ibcast_x(void* buffer, MPI_Count count, MPI _Datatype datatype, int root, MPI _Comm comm, MPI_Info info, MPI_Request *request)

Operations impacted:

1. Collectives Chapter
   MPI_BARRIER, MPI_IBARRIER
   MPI_BCAST, MPI_IBCAST
   MPI_GATHER, MPI_IGATHER
   MPI_GATHERV, MPI_IGATHERV
   MPI_SCATTER, MPI_SCATTERV
   MPI_ALLGATHER, MPI_IALLGATHERV
   MPI_ALLTOALL, MPI_IALLTOALL
   MPI_ALLTOALLV, MPI_IALLTOALLV
   MPI_ALL_TOALLW, MPI_IALLTOALLW
   MPI_REDUCE, MPI_IREDUCE
   MPI_ALLREDUCE, MPI_IALLREDUCE
   MPI_REDUCE_SCATTER_BLOCK, MPI_IREDUCE_SCATTER_BLOCK
   MPI_REDUCE_SCATTER, MPI_IREDUCE_SCATTER
   MPI_SCAN, MPI_ISCAN
   MPI_EXSCAN, MPI_IEXSCAN
   As well as MPI_REDUCE_LOCAL

2. Process Topologies Chapter
   None. This is addressed either by Tickets Non-blocking communicator/file constructors/destructors/etc, respectively to "Groups, Contexts, Communicators, Caching," "Topology," and "I/O" chapters #78 or "WITH_INFO" review across the collective API #84.

Note: No existing functionality is broken, nor is any API deprecated by this ticket.

Tickets #78, #82 proposes new, nonblocking operations. These tickets will separately address the info argument for such new APIs where appropriate. Nothing described in these tickets is impacted by this ticket. Ticket #25 operations already include info keys as well, so this ticket does not impact Ticket #25.

Also, note that this ticket could be folded into Ticket #80 if appropriate.

Changes to the Text

Here is a subset related to topologies (copied from Ticket #84, which is duplicative and is being closed):

MPI_GRAPH_CREATE_WITH_INFO
The "non-dist" graph constructor should be deprecated in favour of the newer (and better) "dist" version(s). Every call to MPI_GRAPH_CREATE can be legally and 'simply' replaced with a call to MPI_DIST_GRAPH_CREATE.

MPI_GRAPH_CREATE requires that each process passes the full (global) communication graph to the call. This limits the scalability of this constructor. With the distributed graph interface, the communication graph is specified in a fully distributed fashion.

In the absence of a proposal to deprecate MPI_GRAPH_CREATE (and thereby obviate the need to maintain/improve it), adding an MPI_INFO argument is one of the advisable changes/improvements.

MPI_CART_CREATE_WITH_INFO
The proposed cartesian topology "with info" constructor seems like a good addition but should probably be part of a more general change to add an MPI_INFO argument to all object constructors that don't already have one. For example, MPI_COMM_SPLIT_WITH_INFO makes a lot of sense too.

--

This ticket can standalone or be approved with Ticket #80. If approved on its own, the info argument would be added to a set of APIs with the suffix _WITH_INFO.

If Ticket #85 is approved, this ticket would be modified to combine with it, as noted above. Ticket #80's changes for "big MPI" would be augmented to add the info argument to each of the abovenamed operations.

Impact on Implementations

1. Incrementally, the info argument of each collective operation would have to be interpreted. It is OK to ignore the info arguments for backward compatibility. Therefore, minimal compliance is nearly trivial. Refactoring the API will only happen once if this is added on to Ticket Big MPI---large-count and displacement support--collective chapter #80 work for refactoring the API.
2. Implementations will be free to use info arguments to offer users enhanced performance and/or predictability of blocking and nonblocking collective APIs. Persistent APIs defined in Ticket Adding Persistent Collective Communication #25 already include this argument.
3. All the impacts of Ticket Big MPI---large-count and displacement support--collective chapter #80 are assumed to be covered by that ticket [if approved].

Impact on Users

Users want to deliver info arguments in particular calls can incrementally modify their programs to use the new API where warranted.

Users opting for the new "big MPI" API already have to make appropriate changes to their code to use functionality added by Ticket #80. This would minimally add MPI_INFO_NULL into an argument slot during such refactoring.

References

See Tickets #25, #76, #78, #80, #83.

[dholmes-epcc-ed-ac-uk]

This gives us the opportunity to use the form MPI_thing_WITH_INFO rather than MPI_thing_X for the BigMPI-and-info API. That is, the _WITH_INFO functions would have both MPI_INFO and MPI_COUNT parameters, whereas the existing API would stand as is with neither of these enhancements. This alleviates concerns over the ambiguity, and possible future use, of _X function names.

[tonyskjellum]

Good point; that is obviously a friendly amendment.

…

On Thu, Mar 8, 2018 at 7:26 AM, Dan Holmes ***@***.***> wrote: This gives us the opportunity to use the form MPI_thing_WITH_INFO rather than MPI_thing_X for the BigMPI-and-info API. That is, the _WITH_INFO functions would have both MPI_INFO and MPI_COUNT parameters, whereas the existing API would stand as is with neither of these enhancements. This alleviates concerns over the ambiguity, and possible future use, of _X function names. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#85 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA38ia2I9aKQHEYn8n5FBRiNa1XoYttlks5tcSN2gaJpZM4SikqW> .

-- Anthony Skjellum, PhD [email protected] Cell: +1-205-807-4968

[tonyskjellum]

We have plenty of work for Austin, and we need to understand the plans and outcomes of Ticket #80 to pursue this ticket efficiently. So I am going to mark its goal as for Barcelona, not Austin. Let's discuss in the WG in Austin.

[tonyskjellum]

We reviewed this in the working group time in Austin. We will discuss this further in the plenary in Austin ahead of presenting this as a reading for the Barcelona meeting.