Merge pull request #9134 from bwbarrett/dist/master-NEWS-sync

dist: Sync NEWS from release branches

Author: bwbarrett

NEWS

@@ -83,6 +83,142 @@ Master (not on release branches yet)
 - Change the default component build behavior to prefer building
   components as part of libmpi.so instead of individual DSOs.
 
+4.1.1 -- April, 2021
+--------------------
+
+- Fix a number of datatype issues, including an issue with
+  improper handling of partial datatypes that could lead to
+  an unexpected application failure.
+- Change UCX PML to not warn about MPI_Request leaks during
+  MPI_FINALIZE by default.  The old behavior can be restored with
+  the mca_pml_ucx_request_leak_check MCA parameter.
+- Reverted temporary solution that worked around launch issues in
+  SLURM v20.11.{0,1,2}. SchedMD encourages users to avoid these
+  versions and to upgrade to v20.11.3 or newer.
+- Updated PMIx to v3.2.2.
+- Fixed configuration issue on Apple Silicon observed with
+  Homebrew. Thanks to François-Xavier Coudert for reporting the issue.
+- Disabled gcc built-in atomics by default on aarch64 platforms.
+- Disabled UCX PML when UCX v1.8.0 is detected. UCX version 1.8.0 has a bug that
+  may cause data corruption when its TCP transport is used in conjunction with
+  the shared memory transport. UCX versions prior to v1.8.0 are not affected by
+  this issue. Thanks to @ksiazekm for reporting the issue.
+- Fixed detection of available UCX transports/devices to better inform PML
+  prioritization.
+- Fixed SLURM support to mark ORTE daemons as non-MPI tasks.
+- Improved AVX detection to more accurately detect supported
+  platforms.  Also improved the generated AVX code, and switched to
+  using word-based MCA params for the op/avx component (vs. numeric
+  big flags).
+- Improved OFI compatibility support and fixed memory leaks in error
+  handling paths.
+- Improved HAN collectives with support for Barrier and Scatter. Thanks
+  to @EmmanuelBRELLE for these changes and the relevant bug fixes.
+- Fixed MPI debugger support (i.e., the MPIR_Breakpoint() symbol).
+  Thanks to @louisespellacy-arm for reporting the issue.
+- Fixed ORTE bug that prevented debuggers from reading MPIR_Proctable.
+- Removed PML uniformity check from the UCX PML to address performance
+  regression.
+- Fixed MPI_Init_thread(3) statement about C++ binding and update
+  references about MPI_THREAD_MULTIPLE.  Thanks to Andreas Lösel for
+  bringing the outdated docs to our attention.
+- Added fence_nb to Flux PMIx support to address segmentation faults.
+- Ensured progress of AIO requests in the POSIX FBTL component to
+  prevent exceeding maximum number of pending requests on MacOS.
+- Used OPAL's mutli-thread support in the orted to leverage atomic
+  operations for object refcounting.
+- Fixed segv when launching with static TCP ports.
+- Fixed --debug-daemons mpirun CLI option.
+- Fixed bug where mpirun did not honor --host in a managed job
+  allocation.
+- Made a managed allocation filter a hostfile/hostlist.
+- Fixed bug to marked a generalized request as pending once initiated.
+- Fixed external PMIx v4.x check.
+- Fixed OSHMEM build with `--enable-mem-debug`.
+- Fixed a performance regression observed with older versions of GCC when
+  __ATOMIC_SEQ_CST is used. Thanks to @BiplabRaut for reporting the issue.
+- Fixed buffer allocation bug in the binomial tree scatter algorithm when
+  non-contiguous datatypes are used. Thanks to @sadcat11 for reporting the issue.
+- Fixed bugs related to the accumulate and atomics functionality in the
+  osc/rdma component.
+- Fixed race condition in MPI group operations observed with
+  MPI_THREAD_MULTIPLE threading level.
+- Fixed a deadlock in the TCP BTL's connection matching logic.
+- Fixed pml/ob1 compilation error when CUDA support is enabled.
+- Fixed a build issue with Lustre caused by unnecessary header includes.
+- Fixed a build issue with IMB LSF workload manager.
+- Fixed linker error with UCX SPML.
+
+4.1.0 -- December, 2020
+-----------------------
+
+- collectives: Add HAN and ADAPT adaptive collectives components.
+  Both components are off by default and can be enabled by specifying
+  "mpirun --mca coll_adapt_priority 100 --mca coll_han_priority 100 ...".
+  We intend to enable both by default in Open MPI 5.0.
+- OMPIO is now the default for MPI-IO on all filesystems, including
+  Lustre (prior to this, ROMIO was the default for Lustre).  Many
+  thanks to Mark Dixon for identifying MPI I/O issues and providing
+  access to Lustre systems for testing.
+- Updates for macOS Big Sur.  Thanks to FX Coudert for reporting this
+  issue and pointing to a solution.
+- Minor MPI one-sided RDMA performance improvements.
+- Fix hcoll MPI_SCATTERV with MPI_IN_PLACE.
+- Add AVX support for MPI collectives.
+- Updates to mpirun(1) about "slots" and PE=x values.
+- Fix buffer allocation for large environment variables.  Thanks to
+  @zrss for reporting the issue.
+- Upgrade the embedded OpenPMIx to v3.2.2.
+- Take more steps towards creating fully Reproducible builds (see
+  https://reproducible-builds.org/).  Thanks Bernhard M. Wiedemann for
+  bringing this to our attention.
+- Fix issue with extra-long values in MCA files.  Thanks to GitHub
+  user @zrss for bringing the issue to our attention.
+- UCX: Fix zero-sized datatype transfers.
+- Fix --cpu-list for non-uniform modes.
+- Fix issue in PMIx callback caused by missing memory barrier on Arm platforms.
+- OFI MTL: Various bug fixes.
+- Fixed issue where MPI_TYPE_CREATE_RESIZED would create a datatype
+  with unexpected extent on oddly-aligned datatypes.
+- collectives: Adjust default tuning thresholds for many collective
+  algorithms
+- runtime: fix situation where rank-by argument does not work
+- Portals4: Clean up error handling corner cases
+- runtime: Remove --enable-install-libpmix option, which has not
+  worked since it was added
+- opal: Disable memory patcher component on MacOS
+- UCX: Allow UCX 1.8 to be used with the btl uct
+- UCX: Replace usage of the deprecated NB API of UCX with NBX
+- OMPIO: Add support for the IME file system
+- OFI/libfabric: Added support for multiple NICs
+- OFI/libfabric: Added support for Scalable Endpoints
+- OFI/libfabric: Added btl for one-sided support
+- OFI/libfabric: Multiple small bugfixes
+- libnbc: Adding numerous performance-improving algorithms
+
+4.0.6 -- March, 2021
+-----------------------
+- Update embedded PMIx to 3.2.2.  This update addresses several
+  MPI_COMM_SPAWN problems.
+- Fix a problem when using Flux PMI and UCX.  Thanks to Sami Ilvonen
+  for reporting and supplying a fix.
+- Fix a problem with MPIR breakpoint being compiled out using PGI
+  compilers.  Thanks to @louisespellacy-arm for reporting.
+- Fix some ROMIO issues when using Lustre.  Thanks to Mark Dixon for
+  reporting.
+- Fix a problem using an external PMIx 4 to build Open MPI 4.0.x.
+- Fix a compile problem when using the enable-timing configure option
+  and UCX.  Thanks to Jan Bierbaum for reporting.
+- Fix a symbol name collision when using the Cray compiler to build
+  Open SHMEM.  Thanks to Pak Lui for reporting and fixing.
+- Correct an issue encountered when building Open MPI under OSX Big Sur.
+  Thanks to FX Coudert for reporting.
+- Various fixes to the OFI MTL.
+- Fix an issue with allocation of sufficient memory for parsing long
+  environment variable values.  Thanks to @zrss for reporting.
+- Improve reproducibility of builds to assist Open MPI packages.
+  Thanks to Bernhard Wiedmann for bringing this to our attention.
+
 4.0.5 -- August, 2020
 ---------------------