Merge pull request #13211 from abouteiller/bugfix/env-thread-level-ignored

bugfix: Setting OMPI_MPI_THREAD_LEVEL to a value different than requested crashes

Author: abouteiller

docs/man-openmpi/man3/MPI_Finalize.3.rst

@@ -5,7 +5,7 @@ MPI_Finalize
 
 .. include_body
 
-:ref:`MPI_Finalize` |mdash| Terminates MPI execution environment.
+:ref:`MPI_Finalize` |mdash| Terminates MPI world model.
 
 .. The following file was automatically generated
 .. include:: ./bindings/mpi_finalize.rst
@@ -18,56 +18,82 @@ OUTPUT PARAMETER
 DESCRIPTION
 -----------
 
-This routine cleans up all MPI states. Once this routine is called, no
-MPI routine (not even MPI_Init) may be called, except for
-:ref:`MPI_Get_version`, :ref:`MPI_Initialized`, and :ref:`MPI_Finalized`. Unless there has
-been a call to :ref:`MPI_Abort`, you must ensure that all pending
-communications involving a process are complete before the process calls
-:ref:`MPI_Finalize`. If the call returns, each process may either continue
-local computations or exit without participating in further
-communication with other processes. At the moment when the last process
-calls :ref:`MPI_Finalize`, all pending sends must be matched by a receive, and
-all pending receives must be matched by a send.
-
-:ref:`MPI_Finalize` is collective over all connected processes. If no processes
-were spawned, accepted, or connected, then this means it is collective
-over MPI_COMM_WORLD. Otherwise, it is collective over the union of all
-processes that have been and continue to be connected.
+This routine finalizes the MPI world model.  If the MPI world model
+has been initialized in an MPI process, it *must* be finalized exactly
+once by invoking this routine during the lifetime of that MPI process.
+This is different than the MPI session model, which can be initialized
+and finalized multiple times in an MPI process.  See
+:ref:`MPI_Session_init` and :ref:`MPI_Session_finalize`.
+
+Unless there has been a call to :ref:`MPI_Abort`, you must
+ensure that all pending communications in the MPI world model
+involving a process are complete before the process calls
+:ref:`MPI_Finalize`. If the call returns, each process may either
+continue local computations or exit without participating in further
+communication with other processes in the MPI world model. At the
+moment when the last process calls :ref:`MPI_Finalize`, all pending
+sends in the MPI world model must be matched by a receive, and all
+pending receives in the MPI world model must be matched by a send.
+
+See `MPI-5.0:11.4.1 <https://www.mpi-forum.org/>`_ for a list of MPI
+functionality that is available (e.g., even when the MPI
+world model has not yet initialized or has already been finalized).
+
+:ref:`MPI_Finalize` is collective over all connected processes. If no
+processes were spawned, accepted, or connected, then this means it is
+collective over ``MPI_COMM_WORLD``. Otherwise, it is collective over
+the union of all processes that have been and continue to be
+connected.
 
 NOTES
 -----
 
-All processes must call this routine before exiting. All processes will
-still exist but may not make any further MPI calls. :ref:`MPI_Finalize`
-guarantees that all local actions required by communications the user
-has completed will, in fact, occur before it returns. However,
-:ref:`MPI_Finalize` guarantees nothing about pending communications that have
-not been completed; completion is ensured only by :ref:`MPI_Wait`, :ref:`MPI_Test`, or
-:ref:`MPI_Request_free` combined with some other verification of completion.
-
-For example, a successful return from a blocking communication operation
-or from :ref:`MPI_Wait` or :ref:`MPI_Test` means that the communication is completed
-by the user and the buffer can be reused, but does not guarantee that
-the local process has no more work to do. Similarly, a successful return
-from :ref:`MPI_Request_free` with a request handle generated by an :ref:`MPI_Isend`
-nullifies the handle but does not guarantee that the operation has
-completed. The :ref:`MPI_Isend` is complete only when a matching receive has
-completed.
-
-If you would like to cause actions to happen when a process finishes,
-attach an attribute to MPI_COMM_SELF with a callback function. Then,
-when :ref:`MPI_Finalize` is called, it will first execute the equivalent of an
-:ref:`MPI_Comm_free` on MPI_COMM_SELF. This will cause the delete callback
-function to be executed on all keys associated with MPI_COMM_SELF in an
-arbitrary order. If no key has been attached to MPI_COMM_SELF, then no
-callback is invoked. This freeing of MPI_COMM_SELF happens before any
-other parts of MPI are affected. Calling :ref:`MPI_Finalized` will thus return
-"false" in any of these callback functions. Once you have done this with
-MPI_COMM_SELF, the results of :ref:`MPI_Finalize` are not specified.
+The MPI session model is different than the MPI world model, and has
+different scopes of availability for MPI functionality.  See
+:ref:`MPI_Session_init` and :ref:`MPI_Session_finalize`.
+
+All processes that initialized the MPI world model must call this
+routine before exiting.  All processes will still exist but may not
+make any further MPI calls in the MPI world model. :ref:`MPI_Finalize`
+guarantees that all local actions required by communications in the
+MPI world model that the user has completed will, in fact, occur
+before it returns. However, :ref:`MPI_Finalize` guarantees nothing
+about pending communications in the MPI world model that have not been
+completed; completion is ensured only by the :ref:`MPI_Wait` and
+:ref:`MPI_Test` variants, or :ref:`MPI_Request_free` combined with
+some other verification of completion.
+
+For example, a successful return from a blocking communication
+operation or from one of the :ref:`MPI_Wait` or :ref:`MPI_Test`
+varients means that the communication is completed by the user and the
+buffer can be reused, but does not guarantee that the local process
+has no more work to do. Similarly, a successful return from
+:ref:`MPI_Request_free` with a request handle generated by an
+:ref:`MPI_Isend` nullifies the handle but does not guarantee that the
+operation has completed. The :ref:`MPI_Isend` is complete only when a
+matching receive has completed.
+
+If you would like to cause actions to happen when a process finalizes the MPI
+world model, attach an attribute to ``MPI_COMM_SELF`` with a callback
+function. Then, when :ref:`MPI_Finalize` is called, it will first
+execute the equivalent of an :ref:`MPI_Comm_free` on
+``MPI_COMM_SELF``. This will cause the delete callback function to be
+executed on all keys associated with ``MPI_COMM_SELF`` in an arbitrary
+order. If no key has been attached to ``MPI_COMM_SELF``, then no
+callback is invoked. This freeing of ``MPI_COMM_SELF`` happens before
+any other parts of the MPI world model are affected. Calling
+:ref:`MPI_Finalized` will thus return ``false`` in any of these
+callback functions. Once you have done this with ``MPI_COMM_SELF``,
+the results of :ref:`MPI_Finalize` are not specified.
 
 ERRORS
 ------
 
 .. include:: ./ERRORS.rst
 
-.. seealso:: :ref:`MPI_Init`
+.. seealso::
+   * :ref:`MPI_Finalized`
+   * :ref:`MPI_Init`
+   * :ref:`MPI_Initialized`
+   * :ref:`MPI_Session_init`
+   * :ref:`MPI_Session_finalize`

docs/man-openmpi/man3/MPI_Finalized.3.rst

@@ -5,28 +5,39 @@ MPI_Finalized
 
 .. include_body
 
-:ref:`MPI_Finalized` |mdash| Checks whether MPI has been finalized
+:ref:`MPI_Finalized` |mdash| Checks whether the MPI world model has been finalized
 
 .. The following file was automatically generated
 .. include:: ./bindings/mpi_finalized.rst
 
 OUTPUT PARAMETER
 ----------------
 
-* ``flag`` : True if MPI was finalized, and false otherwise (logical).
+* ``flag`` : True if the MPI world model was finalized, and false
+  otherwise (logical).
 * ``ierror`` : Fortran only: Error status (integer).
 
 DESCRIPTION
 -----------
 
-This routine may be used to determine whether MPI has been finalized. It
-is one of a small number of routines that may be called before MPI is
-initialized and after MPI has been finalized (:ref:`MPI_Initialized` is
-another).
+This routine may be used to determine whether the MPI world model has
+been finalized.  A different routine |mdash| :ref:`MPI_Initialized`
+|mdash| is used to indicate whether the MPI world model has been
+initialized.
+
+See `MPI-5.0:11.4.1 <https://www.mpi-forum.org/>`_ for a list of MPI
+functionality that is available (e.g., even when the MPI
+world model has not yet initialized or has already been finalized).
 
 ERRORS
 ------
 
 .. include:: ./ERRORS.rst
 
-.. seealso:: :ref:`MPI_Init`
+.. seealso::
+   * :ref:`MPI_Init`
+   * :ref:`MPI_Init_thread`
+   * :ref:`MPI_Finalize`
+   * :ref:`MPI_Finalized`
+   * :ref:`MPI_Session_init`
+   * :ref:`MPI_Session_finalize`

docs/man-openmpi/man3/MPI_Init.3.rst

@@ -6,7 +6,7 @@ MPI_Init
 
 .. include_body
 
-:ref:`MPI_Init` |mdash| Initializes the MPI execution environment
+:ref:`MPI_Init` |mdash| Initializes the MPI world model
 
 .. The following file was automatically generated
 .. include:: ./bindings/mpi_init.rst
@@ -23,35 +23,53 @@ OUTPUT PARAMETER
 DESCRIPTION
 -----------
 
-This routine, or :ref:`MPI_Init_thread`, must be called before most other MPI
-routines are called. There are a small number of errors, such as
-:ref:`MPI_Initialized` and :ref:`MPI_Finalized`. MPI can be initialized at most once;
-subsequent calls to :ref:`MPI_Init` or :ref:`MPI_Init_thread` are erroneous.
+This routine, or :ref:`MPI_Init_thread`, initializes the MPI world
+model.  Either of these routines must be called before MPI
+communication routines are called within the MPI world model.  The MPI
+world model can be initialized at most exactly once in the lifetime of
+an MPI process.  This is different than the MPI session model, which
+can be initialized and finalized multiple times in an MPI process.
+See :ref:`MPI_Session_init` and :ref:`MPI_Session_finalize`.
 
-All MPI programs must contain a call to :ref:`MPI_Init` or :ref:`MPI_Init_thread`.
-Open MPI accepts the C *argc* and *argv* arguments to main, but neither
-modifies, interprets, nor distributes them:
+See `MPI-5.0:11.4.1 <https://www.mpi-forum.org/>`_ for a list of MPI
+functionality that is available (e.g., even when the MPI
+world model has not yet initialized or has already been finalized).
+
+Open MPI's :ref:`MPI_Init` and :ref:`MPI_Init_thread` both accept the
+C *argc* and *argv* arguments to main, but neither modifies,
+interprets, nor distributes them:
 
 .. code-block:: c
 
-   /* declare variables */
-   MPI_Init(&argc, &argv);
-   /* parse arguments */
-   /* main program */
-   MPI_Finalize();
+   #include <mpi.h>
+
+   int main(int argv, char *argv[]) {
+       MPI_Init(&argc, &argv);
+       /* ...body of main MPI pogram... */
+       MPI_Finalize();
+       return 0;
+   }
 
+By default, :ref:`MPI_Init` is effectively equivalent to invoking
+:ref:`MPI_Init_thread` with a *required* value of
+``MPI_THREAD_SINGLE``.  However, if the ``OMPI_MPI_THREAD_LEVEL``
+environment variable is set to a valid value when :ref:`MPI_Init` is
+invoked, it is equivalent to invoking :ref:`MPI_Init_thread` with
+*required* set to the corresponding value of the ``OMPI_MPI_THREAD_LEVEL``
+environment variable.  See :ref:`MPI_Init_thread` for more details.
 
 NOTES
 -----
 
 The Fortran version does not have provisions for *argc* and *argv* and
 takes only IERROR.
 
-The MPI Standard does not say what a program can do before an :ref:`MPI_Init`
-or after an :ref:`MPI_Finalize`. In the Open MPI implementation, it should do
-as little as possible. In particular, avoid anything that changes the
-external state of the program, such as opening files, reading standard
-input, or writing to standard output.
+The MPI Standard does not specify what a program using the MPI world
+model can do before invoking :ref:`MPI_Init` or :ref:`MPI_Init_thread`
+or after invoking :ref:`MPI_Finalize`. In the Open MPI implementation,
+it should do as little as possible. In particular, avoid anything that
+changes the external state of the program, such as opening files,
+reading standard input, or writing to standard output.
 
 
 ERRORS
@@ -64,3 +82,5 @@ ERRORS
    * :ref:`MPI_Initialized`
    * :ref:`MPI_Finalize`
    * :ref:`MPI_Finalized`
+   * :ref:`MPI_Session_finalize`
+   * :ref:`MPI_Session_init`