@@ -198,22 +198,9 @@ using ``--enable-mca-dso`` to selectively build some components as
198
198
DSOs and leave the others included in their respective Open MPI
199
199
libraries.
200
200
201
- .. code :: sh
202
-
203
- # Build all the "accelerator" components as DSOs (all other
204
- # components will default to being built in their respective
205
- # libraries)
206
- shell$ ./configure --enable-mca-dso=accelerator ...
207
-
208
- This allows packaging ``$libdir `` as part of the "main" Open MPI
209
- binary package, but then packaging
210
- ``$libdir/openmpi/mca_accelerator_*.so `` as sub-packages. These
211
- sub-packages may inherit dependencies on the CUDA and/or ROCM
212
- packages, for example. User can always install the "main" Open MPI
213
- binary package, and can install the additional "accelerator" Open MPI
214
- binary sub-package if they actually have accelerator hardware
215
- installed (which will cause the installation of additional
216
- dependencies).
201
+ :ref: `See the section on building accelerator support
202
+ <label-install-packagers-building-accelerator-support-as-dsos>` for a
203
+ practical example where this can be useful.
217
204
218
205
.. _label-install-packagers-gnu-libtool-dependency-flattening :
219
206
@@ -283,3 +270,49 @@ these flattened dependencies, use either of the following mechanisms:
283
270
utilizes compiler and linker flags if they are
284
271
*needed *. All other flags should be the user's /
285
272
packager's choice.
273
+
274
+ .. _label-install-packagers-building-accelerator-support-as-dsos :
275
+
276
+ Building accelerator support as DSOs
277
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
278
+
279
+ If you are building a package that includes support for one or more
280
+ accelerators, it may be desirable to build accelerator-related
281
+ components as DSOs (see the :ref: `static or DSO?
282
+ <label-install-packagers-dso-or-not>` section for details).
283
+
284
+ .. admonition :: Rationale
285
+ :class: tip
286
+
287
+ Accelerator hardware is expensive, and may only be present on some
288
+ compute nodes in an HPC cluster. Specifically: there may not be
289
+ any accelerator hardware on "head" or compile nodes in an HPC
290
+ cluster. As such, invoking Open MPI commands on a "head" node with
291
+ an MPI that was built with static accelerator support but no
292
+ accelerator hardware may fail to launch because of run-time linker
293
+ issues (because the accelerator hardware support libraries are
294
+ likely not present).
295
+
296
+ Building Open MPI's accelerator-related components as DSOs allows
297
+ Open MPI to *try * opening the accelerator components, but proceed
298
+ if those DSOs fail to open due to the lack of support libraries.
299
+
300
+ Use the ``--enable-mca-dso `` command line parameter to Open MPI's
301
+ ``configure `` command can allow packagers to build all
302
+ accelerator-related components as DSO. For example:
303
+
304
+ .. code :: sh
305
+
306
+ # Build all the accelerator-related components as DSOs (all other
307
+ # components will default to being built in their respective
308
+ # libraries)
309
+ shell$ ./configure --enable-mca-dso=btl-smcuda,rcache-rgpusm,rcache-gpusm,accelerator
310
+
311
+ Per the example above, this allows packaging ``$libdir `` as part of
312
+ the "main" Open MPI binary package, but then packaging
313
+ ``$libdir/openmpi/mca_accelerator_*.so `` and the other named
314
+ components as sub-packages. These sub-packages may inherit
315
+ dependencies on the CUDA and/or ROCM packages, for example. The
316
+ "main" package can be installed on all nodes, and the
317
+ accelerator-specific subpackage can be installed on only the nodes
318
+ with accelerator hardware and support libraries.
0 commit comments