Deadlock-free example for nonblocking collectives

[jdinan]

Problem

As reported in [1], MPI 4.1 Example 6.36 contains the incorrect statement:

It is not possible to perform a blocking collective operation on all communicators because there exists no deadlock-free order to invoke them.

Proposal

This can be changed to:

It can be challenging to perform a blocking collective operation on all communicators because some orders of invocation can lead to deadlock.

Impact on Implementations

None

Impact on Users

Clearer example describing nonblocking collectives and why they're useful.

References and Pull Requests

[1] https://stackoverflow.com/questions/77844104/overlapping-communicators-and-collective-operations-in-mpi

[RolfRabenseifner]

I fully agree. The existing text is wrong. The proposal soves the problem.

[devreal]

What purpose does this example serve then? "Here is an example that works either way and it's fairly trivial to get right?" In that case we should remove the example altogether.

I do think there is value in showing an example that would be erroneous with blocking collectives. Swapping the calls on rank 0 would make this example erroneous if we replaced MPI_Iallreduce with MPI_Allreduce:

...
switch ( rank ) {
case 0:
  MPI_Iallreduce (sbuf1 , rbuf1 , count , dtype , MPI_SUM , comm3 , & reqs [0]);
  MPI_Iallreduce (sbuf3 , rbuf3 , count , dtype , MPI_SUM , comm1 , & reqs [1]);
  break ;
case 1:
  MPI_Iallreduce (sbuf1 , rbuf1 , count , dtype , MPI_SUM , comm1 , & reqs [0]);
  MPI_Iallreduce (sbuf2 , rbuf2 , count , dtype , MPI_SUM , comm2 , & reqs [1]);
  break ;
case 2:
  MPI_Iallreduce (sbuf2 , rbuf2 , count , dtype , MPI_SUM , comm2 , & reqs [0]);
  MPI_Iallreduce (sbuf3 , rbuf3 , count , dtype , MPI_SUM , comm3 , & reqs [1]);
  break ;
}
...

The text in question should be slightly rephrased:

It is not possible to perform blocking collective operations in this order. However, nonblocking collective operations can easily be used to achieve this task.