NEWS, HACKING, and README changes for v5.0.0rc1

Updated (lots of) NEWS for v5.0.0rc1.

Signed-off-by: Austen Lauria <awlauria@us.ibm.com>
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>

Author: awlauria

HACKING.md

@@ -7,6 +7,24 @@ source code form, most likely through a developer's tree (i.e., a
 Git clone).
 
 
+## Obtaining Open MPI
+
+Open MPI is available from many distributions, however some users
+prefer to obtain it directly from the Open MPI community via
+prepackaged tarball (see: https://www.open-mpi.org/software/ompi/).
+The Open MPI tarball includes manpages, and openpmix and openprrte
+components, along with an auto-generated configure script.
+
+Some developers prefer to obtain Open MPI by directly cloning it
+from https://github.com/open-mpi/ompi.  It is recommended that users
+who choose to clone the source directly, use the git clone flag
+`--recurse-submodules`, to also obtain the openpmix, and openprrte.
+
+Regardless of how openpmix and openprrte are obtained, the
+configure logic in Open MPI v5.0+ prefer externally installed
+components.  Please see `configure --help` for more details.
+
+
 ## Developer Builds: Compiler Pickyness by Default
 
 If you are building Open MPI from a Git clone (i.e., there is a `.git`

NEWS

@@ -14,11 +14,11 @@ Copyright (c) 2006      Sun Microsystems, Inc.  All rights reserved.
                         Use is subject to license terms.
 Copyright (c) 2006-2017 Los Alamos National Security, LLC.  All rights
                         reserved.
-Copyright (c) 2010-2017 IBM Corporation.  All rights reserved.
+Copyright (c) 2010-2021 IBM Corporation.  All rights reserved.
 Copyright (c) 2012      Oak Ridge National Labs.  All rights reserved.
 Copyright (c) 2012      Sandia National Laboratories.  All rights reserved.
 Copyright (c) 2012      University of Houston. All rights reserved.
-Copyright (c) 2013      NVIDIA Corporation.  All rights reserved.
+Copyright (c) 2013-2021 NVIDIA Corporation.  All rights reserved.
 Copyright (c) 2013-2018 Intel, Inc. All rights reserved.
 Copyright (c) 2018-2021 Amazon.com, Inc. or its affiliates.  All Rights
                         reserved.
@@ -75,13 +75,141 @@ Master (not on release branches yet)
 * launcher.
 **********************************************************************
 
-- Fix rank-by algorithms to properly rank by object and span
-- Do not build Open SHMEM layer when there are no SPMLs available.
-  Currently, this means the Open SHMEM layer will only build if
-  a MXM or UCX library is found.
-- Remove all vestiges of the C/R support
+v5.0.0rc1 -- September, 2021
+--------------------
+- ORTE, the underlying OMPI launcher has been removed, and replaced
+  with PRTE.
+- Reworked how Open MPI integrates with 3rd party packages.
+  The decision was made to stop building 3rd-party packages
+  such as Libevent, HWLOC, PMIx, and PRRTE as MCA components
+  and instead 1) start relying on external libraries whenever
+  possible and 2) Open MPI builds the 3rd party libraries (if needed)
+  as independent libraries, rather than linked into libopen-pal.
+- Update to use PMIx v4.1.1rc2
+- Update to use PRRTE v2.0.1rc2
 - Change the default component build behavior to prefer building
   components as part of libmpi.so instead of individual DSOs.
+- Remove pml/yalla, mxm, mtl/psm, and ikrit components.
+- Remove all vestiges of the C/R support.
+- Various ROMIO v3.4.1 updates.
+- Use Pandoc to generate manpages
+- 32 bit atomics are now only supported via C11 compliant compilers.
+- Do not build Open SHMEM layer when there are no SPMLs available.
+  Currently, this means the Open SHMEM layer will only build if
+  the UCX library is found.
+- Fix rank-by algorithms to properly rank by object and span.
+- Updated the "-mca pml" option to only accept one pml, not a list.
+- vprotocol/pessimist: Updated to support MPI_THREAD_MULLTIPLE.
+- btl/tcp: Updated to use reachability and graph solving for global
+  interface matching. This has been shown to improve MPI_Init()
+  performance under btl/tcp.
+- fs/ime: Fixed compilation errors due to missing header inclusion
+  Thanks to Sylvain Didelot <sdidelot@ddn.com> for finding
+  and fixing this issue.
+- Fixed bug where MPI_Init_thread can give wrong error messages by
+  delaying error reporting until all infrastructure is running.
+- Atomics support removed: S390/s390x, Sparc v9, ARMv4 and ARMv5 CMA
+  support.
+- autogen.pl now supports a "-j" option to run multi-threaded.
+  Users can also use environment variable "AUTOMAKE_JOBS".
+- PMI support has been removed for Open MPI apps.
+- Legacy btl/sm has been removed, and replaced with btl/vader, which
+  was renamed to "btl/sm".
+- Update btl/sm to not use CMA in user namespaces.
+- C++ bindings have been removed.
+- The "--am" and "--amca" options have been deprecated.
+- opal/mca/threads framework added. Currently supports
+  argobots, qthreads, and pthreads. See the --with-threads=x option
+  in configure.
+- Various README.md fixes - thanks to:
+  Yixin Zhang <zhany217@wfu.edu>,
+  Samuel Cho <choss@wfu.edu>,
+  rlangefe <langrc18@wfu.edu>,
+  Alex Ross <rossaj16@wfu.edu>,
+  Sophia Fang <fangq18@wfu.edu>,
+  mitchelltopaloglu <mitchelltopaloglu@gmail.com>,
+  Evstrife <wus217@wfu.edu>, and
+  Hao Tong <tongh18@gemini.deac.wfu.edu> for their
+  contributions.
+- osc/pt2pt: Removed. Users can use osc/rdma + btl/tcp
+  for OSC support using TCP, or other providers.
+- Open MPI now links -levent_core instead of -levent.
+- MPI-4: Added ERRORS_ABORT infrastructure.
+- common/cuda docs: Various fixes. Thanks to
+  Simon Byrne <simonbyrne@gmail.com> for finding and fixing.
+- osc/ucx: Add support for acc_single_intrinsic.
+- Fixed buildrpm.sh "-r" option used for RPM options specification.
+  Thanks to John K. McIver III <john.mciver.iii@gmail.com> for
+  reporting and fixing.
+- configure: Added support for setting the wrapper C compiler.
+  Adds new option "--with-wrapper-cc=" .
+- mpi_f08: Fixed Fortran-8-byte-INTEGER vs. C-4-byte-int issue.
+  Thanks to @ahaichen for reporting the bug.
+- MPI-4: Added support for 'initial error handler'.
+- opal/thread/tsd: Added thread-specific-data (tsd) api.
+- MPI-4: Added error handling for 'unbound' errors to MPI_COMM_SELF.
+- Add missing MPI_Status conversion subroutines:
+  MPI_Status_c2f08(), MPI_Status_f082c(), MPI_Status_f082f(),
+  MPI_Status_f2f08() and the PMPI_* related subroutines.
+- patcher: Removed the Linux component.
+- opal/util: Fixed typo in error string. Thanks to
+  NARIBAYASHI Akira <a.naribayashi@fujitsu.com> for finding
+  and fixing the bug.
+- fortran/use-mpi-f08: Generate PMPI bindings from the MPI bindings.
+- Converted man pages to markdown.
+  Thanks to Fangcong Yin <fyin2@nd.edu> for their contribution
+  to this effort.
+- Fixed ompi_proc_world error string and some comments in pml/ob1.
+  Thanks to Julien EMMANUEL <julien.emmanuel@inria.fr> for
+  finding and fixing these issues.
+- oshmem/tools/oshmem_info: Fixed Fortran keyword issue when
+  compiling param.c. Thanks to Pak Lui <pak.lui@amd.com> for
+  finding and fixing the bug.
+- autogen.pl: Patched libtool.m4 for OSX Big Sur. Thanks to
+  @fxcoudert for reporting the issue.
+- Updgraded to HWLOC v2.4.0.
+- Removed config/opal_check_pmi.m4.
+  Thanks to Zach Osman <zosman@gmu.edu> for the contribution.
+- opal/atomics: Added load-linked, store-conditional atomics for
+  AArch6.
+- Fixed envvar names to OMPI_MCA_orte_precondition_transports.
+  Thanks to Marisa Roman <marisa.roman@cornelisnetworks.com>
+  for the contribution.
+- fcoll/two_phase: Removed the component. All scenerios it was
+  used for has been replaced.
+- btl/uct: Bumped UCX allowed version to v1.9.x.
+- ULFM Fault Tolerance has been added. See README.FT.ULFM.md.
+- Fixed a crash during CUDA initialization.
+  Thanks to Yaz Saito <yasushi.saito@gmail.com> for finding
+  and fixing the bug.
+- Added CUDA support to the OFI MTL.
+- ompio: Added atomicity support.
+- Singleton comm spawn support has been fixed.
+- Autoconf v2.7 support has been updated.
+- fortran: Added check for ISO_FORTRAN_ENV:REAL16. Thanks to
+  Jeff Hammond <jeff_hammond@acm.org> for reporting this issue.
+- Changed the MCA component build style default to static.
+- PowerPC atomics: Force usage of opal/ppc assembly.
+- Removed C++ compiler requirement to build Open MPI.
+- Fixed .la files leaking into wrapper compilers.
+- Fixed bug where the cache line size was not set soon enough in
+  MPI_Init().
+- coll/ucc and scoll/ucc components were added.
+- coll/ucc: Added support for allgather and reduce collective
+  operations.
+- autogen.pl: Fixed bug where it would not ignore all
+  excluded components.
+- Various datatype bugfixes and performance improvements
+- Various pack/unpack bugfixes and performance improvements
+- Fix mmap infinite recurse in memory patcher
+- Fix C to Fortran error code conversions.
+- osc/ucx: Fix data corruption with non-contiguous accumulates
+- Update coll/tuned selection rules
+- Fix non-blocking collective ops
+- btl/portals4: Fix flow control
+- Various oshmem:ucx bugfixes and performance improvements
+- common/ofi: Disable new monitor API until libfabric 1.14.0
+- Fix AVX detection with icc
 
 4.1.1 -- April, 2021
 --------------------

README.md

@@ -94,8 +94,6 @@ The rest of this file contains:
  frameworks](#the-modular-component-architecture-mca)
   * [MPI layer frameworks](#mpi-layer-frameworks)
   * [OpenSHMEM component frameworks](#openshmem-component-frameworks)
-  * [Run-time environment
-    frameworks](#back-end-run-time-environment-rte-component-frameworks)
   * [Miscellaneous frameworks](#miscellaneous-frameworks)
   * [Other notes about frameworks](#framework-notes)
 * [How to get more help](#questions--problems)
@@ -1229,9 +1227,6 @@ Additionally, if a search directory is specified in the form
   so that executables such as `mpicc` and `mpirun` can be found
   without needing to type long path names.
 
-* `--enable-orte-static-ports`:
-   Enable ORTE static ports for TCP OOB (default: enabled).
-
 * `--with-alps`:
   Force the building of for the Cray Alps run-time environment.  If
   Alps support cannot be found, configure will abort.
@@ -2040,16 +2035,17 @@ Open MPI:
 
 * `bml`: BTL management layer
 * `coll`: MPI collective algorithms
-* `fbtl`: file byte transfer layer: abstraction for individual
-   read: collective read and write operations for MPI I/O
-* `fs`: file system functions for MPI I/O
+* `fbtl`: file byte transfer layer: abstraction for individual blocking and non-blocking read and write operations
+* `fcoll`: Collective read and write operations for MPI I/O.
+* `fs`: File system functions for MPI I/O.
+* `hook`: Make calls at various points of MPI process life-cycle.
 * `io`: MPI I/O
 * `mtl`: Matching transport layer, used for MPI point-to-point
    messages on some types of networks
 * `op`: Back end computations for intrinsic MPI_Op operators
 * `osc`: MPI one-sided communications
 * `pml`: MPI point-to-point management layer
-* `rte`: Run-time environment operations
+* `part`: MPI Partitioned communication.
 * `sharedfp`: shared file pointer operations for MPI I/O
 * `topo`: MPI topology routines
 * `vprotocol`: Protocols for the "v" PML
@@ -2064,33 +2060,12 @@ Open MPI:
   point-to-point operations
 * `sshmem`: OpenSHMEM shared memory backing facility
 
-### Back-end run-time environment (RTE) component frameworks:
-
-* `dfs`: Distributed file system
-* `errmgr`: RTE error manager
-* `ess`: RTE environment-specific services
-* `filem`: Remote file management
-* `grpcomm`: RTE group communications
-* `iof`: I/O forwarding
-* `notifier`: System-level notification support
-* `odls`: OpenRTE daemon local launch subsystem
-* `oob`: Out of band messaging
-* `plm`: Process lifecycle management
-* `ras`: Resource allocation system
-* `rmaps`: Resource mapping system
-* `rml`: RTE message layer
-* `routed`: Routing table for the RML
-* `rtc`: Run-time control framework
-* `schizo`: OpenRTE personality framework
-* `state`: RTE state machine
-
 ### Miscellaneous frameworks:
 
 * `allocator`: Memory allocator
 * `backtrace`: Debugging call stack backtrace support
 * `btl`: Point-to-point Byte Transfer Layer
 * `dl`: Dynamic loading library interface
-* `event`: Event library (libevent) versioning support
 * `hwloc`: Hardware locality (hwloc) versioning support
 * `if`: OS IP interface support
 * `installdirs`: Installation directory relocation services
@@ -2100,12 +2075,17 @@ Open MPI:
 * `mpool`: Memory pooling
 * `patcher`: Symbol patcher hooks
 * `pmix`: Process management interface (exascale)
-* `pstat`: Process status
 * `rcache`: Memory registration cache
-* `sec`: Security framework
+* `reachable`: Reachability matrix between endpoints of a given pair of hosts
 * `shmem`: Shared memory support (NOT related to OpenSHMEM)
+* `smsc`: Shared Memory Single Copy
+* `threads`: Thread management and support.
 * `timer`: High-resolution timers
 
+### Back-end run-time environment (PRTE):
+
+See: https://github.com/openpmix/prrte
+
 ### Framework notes
 
 Each framework typically has one or more components that are used at