Skip to content

Commit cc9883d

Browse files
abouteillerabouteiller
authored
Merge pull request #10781 from abouteiller/ulfm/manpages
Adding man pages to document ULFM functions
2 parents 641b6d6 + c65da80 commit cc9883d

11 files changed

+771
-10
lines changed

‎docs/Makefile.am

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -539,6 +539,14 @@ OMPI_MAN3 = \
539539
MPI_Win_wait.3 \
540540
MPI_Wtick.3 \
541541
MPI_Wtime.3 \
542+
MPIX_Comm_ack_failed.3 \
543+
MPIX_Comm_agree.3 \
544+
MPIX_Comm_get_failed.3 \
545+
MPIX_Comm_iagree.3 \
546+
MPIX_Comm_ishrink.3 \
547+
MPIX_Comm_is_revoked.3 \
548+
MPIX_Comm_revoke.3 \
549+
MPIX_Comm_shrink.3 \
542550
MPIX_Query_cuda_support.3 \
543551
MPIX_Query_rocm_support.3 \
544552
OMPI_Affinity_str.3

‎docs/features/ulfm.rst

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -42,24 +42,28 @@ standard draft document.
4242
completion of an MPI operation (error code).
4343
* ``MPIX_ERR_PROC_FAILED_PENDING`` when a potential sender matching a
4444
non-blocking wildcard source receive has failed (error code).
45-
* ``MPIX_ERR_REVOKED`` when one of the ranks in the application has
46-
invoked the ``MPI_Comm_revoke`` operation on the communicator (error
45+
* ``MPIX_ERR_REVOKED`` when the communicator is revoked (error
4746
code).
4847
* ``MPIX_Comm_revoke(MPI_Comm comm)`` Interrupts any communication
49-
pending on the communicator at all ranks (API).
48+
pending on the communicator at all ranks (API). See :ref:`MPIX_Comm_revoke`.
49+
* ``MPIX_Comm_is_revoked(MPI_Comm comm, int *flag)`` Test if a Communicator
50+
is currently revoked (API). See :ref:`MPIX_Comm_is_revoked`.
5051
* ``MPIX_Comm_shrink(MPI_Comm comm, MPI_Comm* newcomm)`` creates a new
5152
communicator where dead processes in comm were removed, and the
5253
remaining processes are renamed to cover all the gaps in the naming
53-
from the original communicator (API).
54+
from the original communicator (API). See :ref:`MPIX_Comm_shrink`,
55+
:ref:`MPIX_Comm_ishrink`.
5456
* ``MPIX_Comm_agree(MPI_Comm comm, int *flag)`` performs a consensus
5557
(i.e. fault tolerant allreduce operation) on flag (with the
5658
operation bitwise AND) (API). Absorbs all new failures, and
57-
propagate the knowledge about failures among the participants.
58-
* ``MPIX_Comm_failure_get_acked(MPI_Comm, MPI_Group*)`` obtains the
59-
group of currently acknowledged failed processes (API).
60-
* ``MPIX_Comm_failure_ack(MPI_Comm)`` acknowledges that the
61-
application intends to ignore the effect of currently known failures
62-
on wildcard receive completions and agreement return values (API).
59+
propagate the knowledge about failures among the participants. see
60+
:ref:`MPIX_Comm_agree`, :ref:`MPIX_Comm_iagree`.
61+
* ``MPIX_Comm_get_failed(MPI_Comm comm, MPI_Group* failedgrp)`` obtains the
62+
group of currently failed processes (API). See :ref:`MPIX_Comm_get_failed`.
63+
* ``MPIX_Comm_ack_failed(MPI_Comm comm, int num_to_ack, int* num_acked)``
64+
acknowledges that the application intends to ignore the effect of currently
65+
known failures on wildcard receive completions and agreement return values
66+
(API). See :ref:`MPIX_Comm_ack_failed`.
6367

6468
Supported Systems
6569
-----------------

‎docs/man-openmpi/man3/MPIX_Comm_ack_failed.3.rst

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
.. _mpix_comm_ack_failed:
2+
3+
MPIX_Comm_ack_failed
4+
====================
5+
.. include_body
6+
7+
:ref:`MPIX_Comm_get_failed` - acknowledge failed processes in a communicator.
8+
9+
This is part of the User Level Fault Mitigation :ref:`ULFM extension <ulfm-label>`.
10+
11+
SYNTAX
12+
------
13+
14+
C Syntax
15+
^^^^^^^^
16+
17+
.. code-block:: c
18+
19+
#include <mpi.h>
20+
#include <mpi-ext.h>
21+
22+
int MPIX_Comm_ack_failed(MPI_Comm comm, int num_to_ack, int *num_acked)
23+
24+
Fortran Syntax
25+
^^^^^^^^^^^^^^
26+
27+
.. code-block:: fortran
28+
29+
USE MPI
30+
USE MPI_EXT
31+
! or the older form: INCLUDE 'mpif.h'
32+
33+
MPIX_COMM_ACK_FAILED(COMM, NUM_TO_ACK, NUM_ACKED, IERROR)
34+
INTEGER COMM, NUM_TO_ACK, NUM_ACKED, IERROR
35+
36+
Fortran 2008 Syntax
37+
^^^^^^^^^^^^^^^^^^^
38+
39+
.. code-block:: fortran
40+
41+
USE mpi_f08
42+
USE mpi_ext_f08
43+
44+
MPIX_Comm_ack_failed(comm, num_to_ack, num_acked, ierror)
45+
TYPE(MPI_Comm), INTENT(IN) :: comm
46+
INTEGER, INTENT(IN) :: num_to_ack
47+
INTEGER, INTENT(OUT) :: num_acked
48+
INTEGER, OPTIONAL, INTENT(OUT) :: ierror
49+
50+
INPUT PARAMETERS
51+
----------------
52+
* ``comm``: Communicator (handle).
53+
* ``num_to_ack``: maximum number of process failures to acknowledge in *comm* (integer)
54+
55+
OUTPUT PARAMETERS
56+
-----------------
57+
* ``num_acked``: number of acknowledged failures in *comm* (integer).
58+
* ``ierror``: Fortran only: Error status (integer).
59+
60+
DESCRIPTION
61+
-----------
62+
63+
his local operation gives the users a way to **acknowledge**
64+
locally notified failures on *comm*. The operation acknowledges the first
65+
*num_to_ack* process failures on *comm*, that is, it acknowledges the
66+
failure of members with a rank lower than *num_to_ack* in the group that
67+
would be produced by a concurrent call to :ref:`MPIX_Comm_get_failed` on
68+
the same *comm*.
69+
70+
The operation also sets the value of *num_acked* to the current number of
71+
acknowledged process failures in *comm*, that is, a process failure has been
72+
acknowledged on *comm* if and only if the rank of the process is lower than
73+
*num_acked* in the group that would be produced by a subsequent call to
74+
:ref:`MPIX_Comm_get_failed` on the same *comm*.
75+
76+
*num_acked* can be larger than *num_to_ack* when process failures have been
77+
acknowledged in a prior call to :ref:`MPIX_Comm_ack_failed`.
78+
79+
EFFECT OF ACKNOWLEDGING FAILURES
80+
--------------------------------
81+
82+
After an MPI process failure is acknowledged on *comm*, unmatched
83+
MPI_ANY_SOURCE receive operations on the same *comm* that would have raised
84+
an error of class MPIX_ERR_PROC_FAILED_PENDING proceed without further raising
85+
errors due to this acknowledged failure.
86+
87+
Also, :ref:`MPIX_Comm_agree` on the same *comm* will not raise an error of
88+
class MPI_ERR_PROC_FAILED due to this acknowledged failure.
89+
90+
USAGE PATTERNS
91+
--------------
92+
93+
One may query, without side effect, for the number of currently aknowledged
94+
process failures *comm* by supplying 0 in *num_to_ack*.
95+
96+
Conversely, one may unconditionally acknowledge all currently known process
97+
failures in *comm* by supplying the size of the group of *comm* in *num_to_ack*.
98+
99+
Note that the number of acknowledged processes, as returned in *num_acked*,
100+
can be smaller or larger than the value supplied in *num_to_ack*; It is
101+
however never larger than the size of the group returned by a subsequent call
102+
to :ref:`MPIX_Comm_get_failed`.
103+
104+
EFFECT ON COLLECTIVE OPERATIONS
105+
-------------------------------
106+
107+
Calling :ref:`MPIX_Comm_ack_failed` on a communicator with failed MPI
108+
processes has no effect on collective operations (except for :ref:`MPIX_Comm_agree`).
109+
If a collective operation would raise an error due to the communicator
110+
containing a failed process it will continue to raise an error even after
111+
the failure has been acknowledged. In order to use collective operations
112+
between MPI processes of a communicator that contains failed MPI processes,
113+
users should create a new communicator (e.g., by calling :ref:`MPIX_Comm_shrink`).
114+
115+
WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
116+
------------------------------------------
117+
118+
When the communicator is an inter-communicator, the failures of members
119+
in both the local and the remote groups of *comm* are acknowledged.
120+
121+
ERRORS
122+
------
123+
124+
.. include:: ./ERRORS.rst
125+
126+
.. seealso::
127+
* :ref:`MPIX_Comm_get_failed`
128+
* :ref:`MPIX_Comm_agree`

‎docs/man-openmpi/man3/MPIX_Comm_agree.3.rst

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
.. _mpix_comm_agree:
2+
3+
MPIX_Comm_agree
4+
===============
5+
.. include_body
6+
7+
:ref:`MPIX_Comm_agree`, :ref:`MPIX_Comm_iagree` - Agree on a flag value
8+
from all live processes and distributes the result back to all live
9+
processes, even after process failures.
10+
11+
This is part of the User Level Fault Mitigation :ref:`ULFM extension <ulfm-label>`.
12+
13+
SYNTAX
14+
------
15+
16+
C Syntax
17+
^^^^^^^^
18+
19+
.. code-block:: c
20+
21+
#include <mpi.h>
22+
#include <mpi-ext.h>
23+
24+
int MPIX_Comm_agree(MPI_Comm comm, int *flag)
25+
26+
int MPIX_Comm_iagree(MPI_Comm comm, int *flag, MPI_Request *request)
27+
28+
29+
Fortran Syntax
30+
^^^^^^^^^^^^^^
31+
32+
.. code-block:: fortran
33+
34+
USE MPI
35+
USE MPI_EXT
36+
! or the older form: INCLUDE 'mpif.h'
37+
38+
MPIX_COMM_AGREE(COMM, FLAG, IERROR)
39+
INTEGER COMM, FLAG, IERROR
40+
41+
MPIX_COMM_IAGREE(COMM, FLAG, REQUEST, IERROR)
42+
INTEGER COMM, FLAG, REQUEST, IERROR
43+
44+
45+
Fortran 2008 Syntax
46+
^^^^^^^^^^^^^^^^^^^
47+
48+
.. code-block:: fortran
49+
50+
USE mpi_f08
51+
USE mpi_ext_f08
52+
53+
MPIX_Comm_agree(comm, flag, ierror)
54+
TYPE(MPI_Comm), INTENT(IN) :: comm
55+
INTEGER, INTENT(INOUT) :: flag
56+
INTEGER, OPTIONAL, INTENT(OUT) :: ierror
57+
58+
MPIX_COMM_IAGREE(COMM, FLAG, REQUEST, IERROR)
59+
TYPE(MPI_Comm), INTENT(IN) :: comm
60+
INTEGER, INTENT(INOUT), ASYNCHRONOUS :: flag
61+
TYPE(MPI_Request), INTENT(OUT) :: request
62+
INTEGER, OPTIONAL, INTENT(OUT) :: ierror
63+
64+
INPUT PARAMETERS
65+
----------------
66+
* ``comm``: Communicator (handle).
67+
* ``flag``: Binary flags (integer).
68+
69+
OUTPUT PARAMETERS
70+
-----------------
71+
* ``flag``: Reduced binary flags (integer).
72+
* ``request``: Request (handle, non-blocking only).
73+
* ``ierror``: Fortran only: Error status (integer).
74+
75+
DESCRIPTION
76+
-----------
77+
78+
This collective communication agrees on the integer value *flag* and
79+
(implicitly) on the group of failed processes in *comm*.
80+
81+
On completion, all non-failed MPI processes have agreed to set the
82+
output integer value of *flag* to the result of a *bitwise AND*
83+
operation over the contributed input values of *flag*.
84+
85+
:ref:`MPIX_Comm_iagree` is the non-blocking variant of :ref:`MPIX_Comm_agree`.
86+
87+
PROCESS FAILURES
88+
----------------
89+
90+
When an MPI process fails before contributing to the agree operation,
91+
the *flag* is computed ignoring its contribution, and the operation
92+
raises an error of class MPIX_ERR_PROC_FAILED.
93+
94+
When an error of class MPIX_ERR_PROC_FAILED is raised, it is consistently
95+
raised at all MPI processes in the group(s) of *comm*.
96+
97+
After :ref:`MPIX_Comm_agree` raised an error of class MPIX_ERR_PROC_FAILED,
98+
the group produced by a subsequent call to :ref:`MPIX_Comm_get_failed` on
99+
*comm* contains every MPI process that didn't contribute to the
100+
computation of *flag*.
101+
102+
WHEN THE COMMUNICATOR CONTAINS ACKNOWLEDGED FAILURES
103+
----------------------------------------------------
104+
105+
If **all** MPI processes in the group of *comm* have acknowledged the failure
106+
of an MPI process (using :ref:`MPIX_Comm_ack_failed`) prior to the call to
107+
:ref:`MPIX_Comm_agree` (or :ref:`MPIX_Comm_iagree`), the MPIX_ERR_PROC_FAILED
108+
error is not raised when the output value of *flag* ignores the
109+
contribution of that failed process. Note that this is an uniform property:
110+
if a non-contributing process is found to be not-acknowledged at any live
111+
process in *comm*, all processes raise an error of class MPIX_ERR_PROC_FAILED.
112+
113+
**Example 1:** Using a combination of :ref:`MPIX_Comm_ack_failed` and
114+
:ref:`MPIX_Comm_agree` users can propagate and synchronize the knowledge
115+
of failures across all MPI processes in *comm*.
116+
117+
.. code-block:: c
118+
119+
Comm_get_failed_consistent(MPI_Comm c, MPI_Group * g) {
120+
int rc; int T=1;
121+
int size; int num_acked;
122+
MPI_Group gf;
123+
int ranges[3] = {0, 0, 1};
124+
125+
MPI_Comm_size(c, &size);
126+
127+
do {
128+
/* this routine is not pure: calling MPI_Comm_ack_failed
129+
* affects the state of the communicator c */
130+
MPIX_Comm_ack_failed(c, size, &num_acked);
131+
/* we simply ignore the T value in this example */
132+
rc = MPIX_Comm_agree(c, &T);
133+
} while( rc != MPI_SUCCESS );
134+
/* after this loop, MPIX_Comm_agree has returned MPI_SUCCESS at
135+
* all processes, so all processes have Acknowledged the same set of
136+
* failures. Let's get that set of failures in the g group. */
137+
if( 0 == num_acked ) {
138+
*g = MPI_GROUP_EMPTY;
139+
}
140+
else {
141+
MPIX_Comm_get_failed(c, &gf);
142+
ranges[1] = num_acked - 1;
143+
MPI_Group_range_incl(gf, 1, ranges, g);
144+
MPI_Group_free(&gf);
145+
}
146+
}
147+
148+
WHEN THE COMMUNICATOR IS REVOKED
149+
--------------------------------
150+
151+
This function never raises an error of class MPIX_ERR_REVOKED.
152+
The defined semantics of :ref:`MPIX_Comm_agree` are maintained when *comm*
153+
is revoked, or when the group of *comm* contains failed MPI processes.
154+
In particular, :ref:`MPIX_Comm_agree` is a collective operation, even
155+
when *comm* is revoked.
156+
157+
WHEN COMMUNICATOR IS AN INTER-COMMUNICATOR
158+
------------------------------------------
159+
160+
When the communicator is an inter-communicator, the value of *flag* is
161+
a *bitwise AND* operation over the values contributed by the remote
162+
group.
163+
164+
When an error of class MPIX_ERR_PROC_FAILED is raised, it is consistently
165+
raised at all MPI processes in the group(s) of *comm*, that is, both
166+
the local and remote groups of the inter-communicator.
167+
168+
ERRORS
169+
------
170+
171+
.. include:: ./ERRORS.rst
172+
173+
.. seealso::
174+
* :ref:`MPIX_Comm_is_revoked`
175+
* :ref:`MPIX_Comm_ack_failed`

0 commit comments

Comments
 (0)