Skip to content

Commit 8704b52

Browse files
authored
Merge pull request #87603 from dfitzmau/OCPBUGS-48847
OCPBUGS#48847: Documented IPoIB support for nmstate
2 parents e2f755e + 0318476 commit 8704b52

File tree

3 files changed

+68
-1
lines changed

3 files changed

+68
-1
lines changed

installing/installing_bare_metal/ipi/ipi-install-post-installation-configuration.adoc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@ include::modules/nw-enabling-a-provisioning-network-after-installation.adoc[leve
1717
// Creating a manifest object that includes a customized `br-ex` bridge
1818
include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset=+1]
1919

20+
// Creating an InfiniBand interface on nodes
21+
include::modules/virt-creating-infiniband-interface-on-nodes.adoc[leveloffset=+1]
22+
2023
// Services for a user-managed load balancer
2124
include::modules/nw-osp-services-external-load-balancer.adoc[leveloffset=+1]
2225

modules/creating-manifest-file-customized-br-ex-bridge.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ endif::[]
1717

1818
:_mod-docs-content-type: PROCEDURE
1919
[id="creating-manifest-file-customized-br-ex-bridge_{context}"]
20-
== Creating a manifest object that includes a customized `br-ex` bridge
20+
= Creating a manifest object that includes a customized `br-ex` bridge
2121

2222
ifndef::postinstall-bare-metal-ipi,postinstall-bare-metal-upi[]
2323
As an alternative to using the `configure-ovs.sh` shell script to set a `br-ex` bridge on a bare-metal platform, you can create a `MachineConfig` object that includes an NMState configuration file. The NMState configuration file creates a customized `br-ex` bridge network configuration on each node in your cluster.
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * installing/installing_bare_metal/ipi/ipi-install-post-installation-configuration.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="virt-creating-infiniband-interface-on-nodes_{context}"]
7+
= Creating an IP over InfiniBand interface on nodes
8+
9+
On the {product-title} web console, you can install a Red{nbsp}Hat certified third-party Operator, such as the NVIDIA Network Operator, that supports InfiniBand (IPoIB) mode. Typically, you would use the third-party Operator with other vendor infrastructure to manage resources in an {product-title} cluster. To create an IPoIB interface on nodes in your cluster, you must define an InfiniBand (IPoIB) interface in a `NodeNetworkConfigurationPolicy` (NNCP) manifest file.
10+
11+
[IMPORTANT]
12+
====
13+
The {product-title} documentation describes defining only the IPoIB interface configuration in a `NodeNetworkConfigurationPolicy` (NNCP) manifest file. You must refer to the NVIDIA and other third-party vendor documentation for the majority of the configuring steps. Red{nbsp}Hat support does not extend to anything external to the NNCP configuration.
14+
15+
For more information about the NVIDIA Operator, see link:https://docs.nvidia.com/networking/display/kubernetes2410/getting+started+with+red+hat+openshift[Getting Started with Red{nbsp}Hat OpenShift] (NVIDIA Docs Hub).
16+
====
17+
18+
.Prerequisites
19+
20+
* You installed a Red{nbsp}Hat certified third-party Operator that supports an IPoIB interface.
21+
22+
23+
.Procedure
24+
25+
. Create or edit a `NodeNetworkConfigurationPolicy` (NNCP) manifest file, and then specify an IPoIB interface in the file.
26+
+
27+
28+
[source,yaml]
29+
----
30+
apiVersion: nmstate.io/v1
31+
kind: NodeNetworkConfigurationPolicy
32+
metadata:
33+
name: worker-0-ipoib
34+
spec:
35+
# ...
36+
interfaces:
37+
- description: ""
38+
infiniband:
39+
mode: datagram <1>
40+
pkey: "0xffff" <2>
41+
ipv4:
42+
address:
43+
- ip: 100.125.3.4
44+
prefix-length: 16
45+
dhcp: false
46+
enabled: true
47+
ipv6:
48+
enabled: false
49+
name: ibp27s0
50+
state: up
51+
type: infiniband <3>
52+
# ...
53+
----
54+
<1> `datagram` is the default mode for an IPoIB interface, and this mode improves optimizes performance and latency. `connected` mode is a supported mode but consider only using this mode when you need to adjust the maximum transmission unit (MTU) value to improve node connectivity with surrounding network devices.
55+
<2> Supports a string or an integer value. The parameter defines the protection key, or _P-key_, for the interface for the purposes of authentication and encrypted communications with a third-party vendor, such as NVIDIA. Values `None` and `0xffff` indicate the protection key for the base interface in an InfiniBand system.
56+
<3> Sets the type of interface to `infiniband `.
57+
58+
. Apply the NNCP configuration to each node in your cluster by running the following command. The Kubernetes NMState Operator can then create an IPoIB interface on each node.
59+
+
60+
[source,yaml]
61+
----
62+
$ oc apply -f <nncp_file_name> <1>
63+
----
64+
<1> Replace `<nncp_file_name>` with the name of your NNCP file.

0 commit comments

Comments
 (0)