Skip to content

Commit 943d8ec

Browse files
dfitzmaudfitzmau
authored
Merge pull request #92731 from dfitzmau/OSDOCS-13333
OSDOCS-13333: Add perf scale considerations to whereabouts CNI plugin…
2 parents e09888a + 1e1933f commit 943d8ec

File tree

7 files changed

+278
-86
lines changed

7 files changed

+278
-86
lines changed

‎modules/nw-multus-ipam-object.adoc

Lines changed: 0 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -195,89 +195,3 @@ The following JSON example describes the configuration p for dynamic IP address
195195
}
196196
}
197197
----
198-
199-
[id="nw-multus-whereabouts_{context}"]
200-
== Dynamic IP address assignment configuration with Whereabouts
201-
202-
The Whereabouts CNI plugin allows the dynamic assignment of an IP address to a secondary network without the use of a DHCP server.
203-
204-
The Whereabouts CNI plugin also supports overlapping IP address ranges and configuration of the same CIDR range multiple times within separate `NetworkAttachmentDefinition` CRDs. This provides greater flexibility and management capabilities in multi-tenant environments.
205-
206-
[id="dynamic-ip-address-assignment-objects_{context}"]
207-
=== Dynamic IP address configuration objects
208-
209-
The following table describes the configuration objects for dynamic IP address assignment with Whereabouts:
210-
211-
.`ipam` whereabouts configuration object
212-
[cols=".^2,.^2,.^6",options="header"]
213-
|====
214-
|Field|Type|Description
215-
216-
|`type`
217-
|`string`
218-
|The IPAM address type. The value `whereabouts` is required.
219-
220-
|`range`
221-
|`string`
222-
|An IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
223-
224-
|`exclude`
225-
|`array`
226-
|Optional: A list of zero or more IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.
227-
228-
|`network_name`
229-
|`string`
230-
| Optional: Helps ensure that each group or domain of pods gets its own set of IP addresses, even if they share the same range of IP addresses. Setting this field is important for keeping networks separate and organized, notably in multi-tenant environments.
231-
232-
|====
233-
234-
[id="dynamic-ip-address-assignment-whereabouts_{context}"]
235-
=== Dynamic IP address assignment configuration that uses Whereabouts
236-
237-
The following example shows a dynamic address assignment configuration that uses Whereabouts:
238-
239-
.Whereabouts dynamic IP address assignment
240-
[source,json]
241-
----
242-
{
243-
"ipam": {
244-
"type": "whereabouts",
245-
"range": "192.0.2.192/27",
246-
"exclude": [
247-
"192.0.2.192/30",
248-
"192.0.2.196/32"
249-
]
250-
}
251-
}
252-
----
253-
254-
[id="dynamic-ip-address-assignment-whereabouts-overlapping-ip-ranges_{context}"]
255-
=== Dynamic IP address assignment that uses Whereabouts with overlapping IP address ranges
256-
257-
The following example shows a dynamic IP address assignment that uses overlapping IP address ranges for multi-tenant networks.
258-
259-
.NetworkAttachmentDefinition 1
260-
[source,json]
261-
----
262-
{
263-
"ipam": {
264-
"type": "whereabouts",
265-
"range": "192.0.2.192/29",
266-
"network_name": "example_net_common", <1>
267-
}
268-
}
269-
----
270-
<1> Optional. If set, must match the `network_name` of `NetworkAttachmentDefinition 2`.
271-
272-
.NetworkAttachmentDefinition 2
273-
[source,json]
274-
----
275-
{
276-
"ipam": {
277-
"type": "whereabouts",
278-
"range": "192.0.2.192/24",
279-
"network_name": "example_net_common", <1>
280-
}
281-
}
282-
----
283-
<1> Optional. If set, must match the `network_name` of `NetworkAttachmentDefinition 1`.

‎modules/nw-multus-whereabouts-fast-ipam.adoc

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/multiple_networks/configuring-additional-network.adoc
4+
// * networking/hardware_networks/configuring-sriov-net-attach.adoc
5+
// * virt/vm_networking/virt-connecting-vm-to-sriov.adoc
6+
7+
:_mod-docs-content-type: PROCEDURE
8+
[id="nw-multus-whereabouts-fast-ipam_{context}"]
9+
= Fast IPAM configuration for the Whereabouts IPAM CNI plugin
10+
11+
Wherabouts is an IP Address Management (IPAM) Container Network Interface (CNI) plugin that assigns IP addresses at a cluster-wide level. Whereabouts does not require a Dynamic Host Configuration Protocol (DHCP) server.
12+
13+
A typical Wherabouts workflow is described as follows:
14+
15+
. Whereabouts takes an address range in classless inter-domain routing (CIDR) notation, such as `192.168.2.0/24`, and assigns IP addresses within that range, such as `192.168.2.1` to `192.168.2.254`.
16+
. Whereabouts assigns an IP address, the lowest value address in a CIDR range, to a pod and tracks the IP address in a data store for the lifetime of that pod.
17+
. When the pod is removed, Whereabouts frees the address from the pod so that the address is available for assignment.
18+
19+
To improve the performance of Whereabouts, especially if nodes in your cluster run a high amount of pods, you can enable the Fast IPAM feature.
20+
21+
:FeatureName: Fast IPAM
22+
include::snippets/technology-preview.adoc[leveloffset=+1]
23+
24+
The Fast IPAM feature uses `nodeslicepools`, which are managed by the Whereabouts Controller, to optimize IP allocation for nodes.
25+
26+
.Prerequisites
27+
28+
* You added the `whereabouts-shim` configuration to the `Network.operator.openshift.io` custom resource (CR), so that the Cluster Network Operator (CNO) can deploy the Whereabouts Controller. See "Creating a Whereabouts reconciler daemon set".
29+
* For the Fast IPAM feature to work, ensure that the `NetworkAttachmentDefinition` (NAD) and the pod exist in the same `openshift-multus` namesapace.
30+
31+
.Procedure
32+
33+
. Confirm that the Whereabouts Controller is running by entering the following command.
34+
+
35+
[source,terminal]
36+
----
37+
$ oc get pods -n openshift-multus | grep controller
38+
----
39+
+
40+
.Example output
41+
[source,terminal]
42+
----
43+
multus-admission-controller-d89bc96f-gbf7s 2/2 Running 0 6h3m
44+
...
45+
----
46+
+
47+
[IMPORTANT]
48+
====
49+
If the Whereabouts Controller is not running, the Fast IPAM does not work.
50+
====
51+
52+
. Create a NAD file for your cluster and add the Fast IPAM details to the file:
53+
+
54+
.Example NAD file with a Fast IPAM configuration
55+
[source,yaml,subs="attributes+,quotes"]
56+
----
57+
apiVersion: "k8s.cni.cncf.io/v1"
58+
kind: NetworkAttachmentDefinition
59+
metadata:
60+
name: wb-ipam
61+
namespace: openshift-multus <1>
62+
spec:
63+
config: {
64+
"cniVersion": "0.3.0",
65+
"name": "wb-ipam-cni-name", <2>
66+
"type": "bridge",
67+
"bridge": "cni0",
68+
"ipam": {
69+
"type": "whereabouts", <3>
70+
"range": "10.5.0.0/20", <4>
71+
"node_slice_size": "/24" <5>
72+
}
73+
}
74+
# ...
75+
----
76+
<1> The namespace where CNO deploys the NAD.
77+
<2> The name of the Whereabouts IPAM CNI plugin.
78+
<3> The type of IPAM CNI plugin: `whereabouts`.
79+
<4> The IP address range for the IP pool that the Whereabouts IPAM CNI plugin uses for allocating IP addresses to pods.
80+
<5> Sets the slice size of IP addresses available to each node.
81+
82+
. Add the Whereabouts IPAM CNI plugin annotation details to the YAML file for the pod:
83+
+
84+
[source,yaml,subs="attributes+,quotes"]
85+
----
86+
apiVersion: v1
87+
kind: Pod
88+
metadata:
89+
name: <pod_name> <1>
90+
annotations:
91+
k8s.v1.cni.cncf.io/networks: openshift-multus/wb-ipam <2>
92+
spec:
93+
containers:
94+
- name: samplepod <3>
95+
command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"] <4>
96+
image: alpine
97+
# ...
98+
----
99+
<1> The name of the pod.
100+
<2> The annotation details that references the Whereabouts IPAM CNI plugin name that exists in the `openshift-multus` namespace.
101+
<3> The name of the container for the pod.
102+
<4> Defines the entry point for the container and controls the behavior of the container in the Whereabouts IPAM CNI plugin.
103+
104+
. Apply the NAD file configuration to pods that exist on nodes that run in your cluster:
105+
+
106+
[source,terminal]
107+
----
108+
$ oc create -f <NAD_file_name>.yaml
109+
----
110+
111+
.Verification
112+
113+
. Show the IP address details of the pod by entering the following command:
114+
+
115+
[source,terminal,subs="attributes+,quotes"]
116+
----
117+
$ oc describe pod <pod_name>
118+
----
119+
+
120+
.Example output
121+
+
122+
[source,terminal]
123+
----
124+
...
125+
IP: 192.168.2.0
126+
IPs:
127+
IP: 192.168.2.0
128+
Containers:
129+
samplepod:
130+
Container ID: docker://<image_name>
131+
Image: <app_name>:v1
132+
Image ID:
133+
...
134+
----
135+
136+
. Access the pod and confirm its interfaces by entering the following command:
137+
+
138+
[source,terminal]
139+
----
140+
$ oc exec <pod_name> -- ip a
141+
----
142+
+
143+
.Example output
144+
[source,terminal]
145+
----
146+
...
147+
3: net1@if23: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
148+
link/ether 82:01:98:e5:0c:b7 brd ff:ff:ff:ff:ff:ff
149+
inet 192.168.2.0/24 brd 10.10.0.255 scope global net1 <1>
150+
valid_lft forever preferred_lft forever
151+
inet6 fe80::8001:98ff:fee5:cb7/64 scope link
152+
valid_lft forever preferred_lft forever
153+
...
154+
----
155+
<1> Pod is attached to the `192.168.2.1` IP address on the `net1` interface as expected.
156+
157+
. Check that the node selector pool exists in the `openshift-multus` namespace by entering the following command:
158+
+
159+
[source,terminal]
160+
----
161+
$ oc get nodeslicepool -n openshift-multus
162+
----
163+
+
164+
.Example output
165+
[source,terminal]
166+
----
167+
NAME AGE
168+
nodeslicepool 32m
169+
----

‎modules/nw-multus-whereabouts.adoc

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/multiple_networks/configuring-additional-network.adoc
4+
// * networking/hardware_networks/configuring-sriov-net-attach.adoc
5+
// * virt/vm_networking/virt-connecting-vm-to-sriov.adoc
6+
7+
:_mod-docs-content-type: REFERENCE
8+
[id="nw-multus-whereabouts_{context}"]
9+
= Dynamic IP address assignment configuration with Whereabouts
10+
11+
The Whereabouts CNI plugin helps the dynamic assignment of an IP address to a secondary network without the use of a DHCP server.
12+
13+
The Whereabouts CNI plugin also supports overlapping IP address ranges and configuration of the same CIDR range multiple times within separate `NetworkAttachmentDefinition` CRDs. This provides greater flexibility and management capabilities in multi-tenant environments.
14+
15+
[id="dynamic-ip-address-assignment-objects_{context}"]
16+
== Dynamic IP address configuration parameters
17+
18+
The following table describes the configuration objects for dynamic IP address assignment with Whereabouts:
19+
20+
.`ipam` whereabouts configuration parameters
21+
[cols=".^2,.^2,.^6",options="header"]
22+
|====
23+
|Field|Type|Description
24+
25+
|`type`
26+
|`string`
27+
|The IPAM address type. The value `whereabouts` is required.
28+
29+
|`range`
30+
|`string`
31+
|An IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
32+
33+
|`exclude`
34+
|`array`
35+
|Optional: A list of zero or more IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.
36+
37+
|`network_name`
38+
|`string`
39+
| Optional: Helps ensure that each group or domain of pods gets its own set of IP addresses, even if they share the same range of IP addresses. Setting this field is important for keeping networks separate and organized, notably in multi-tenant environments.
40+
41+
|====
42+
43+
[id="dynamic-ip-address-assignment-whereabouts_{context}"]
44+
== Dynamic IP address assignment configuration with Whereabouts that excludes IP address ranges
45+
46+
The following example shows a dynamic address assignment configuration in a NAD file that uses Whereabouts:
47+
48+
.Whereabouts dynamic IP address assignment that excludes specific IP address ranges
49+
[source,json]
50+
----
51+
{
52+
"ipam": {
53+
"type": "whereabouts",
54+
"range": "192.0.2.192/27",
55+
"exclude": [
56+
"192.0.2.192/30",
57+
"192.0.2.196/32"
58+
]
59+
}
60+
}
61+
----
62+
63+
[id="dynamic-ip-address-assignment-whereabouts-overlapping-ip-ranges_{context}"]
64+
== Dynamic IP address assignment that uses Whereabouts with overlapping IP address ranges
65+
66+
The following example shows a dynamic IP address assignment that uses overlapping IP address ranges for multi-tenant networks.
67+
68+
.NetworkAttachmentDefinition 1
69+
[source,json]
70+
----
71+
{
72+
"ipam": {
73+
"type": "whereabouts",
74+
"range": "192.0.2.192/29",
75+
"network_name": "example_net_common", <1>
76+
}
77+
}
78+
----
79+
<1> Optional. If set, must match the `network_name` of `NetworkAttachmentDefinition 2`.
80+
81+
.NetworkAttachmentDefinition 2
82+
[source,json]
83+
----
84+
{
85+
"ipam": {
86+
"type": "whereabouts",
87+
"range": "192.0.2.192/24",
88+
"network_name": "example_net_common", <1>
89+
}
90+
}
91+
----
92+
<1> Optional. If set, must match the `network_name` of `NetworkAttachmentDefinition 1`.

‎networking/hardware_networks/configuring-sriov-ib-attach.adoc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ include::modules/nw-multus-configure-dualstack-ip-address.adoc[leveloffset=+2]
1919
// Configuration of IP address assignment for a network attachment
2020
include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
2121

22+
// Dynamic IP address assignment configuration with Whereabouts
23+
include::modules/nw-multus-whereabouts.adoc[leveloffset=+2]
24+
2225
// Configuring SR-IOV additional network
2326
include::modules/nw-sriov-network-attachment.adoc[leveloffset=+1]
2427

‎networking/hardware_networks/configuring-sriov-net-attach.adoc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ include::modules/nw-multus-configure-dualstack-ip-address.adoc[leveloffset=+2]
1919
// Configuration of IP address assignment for a network attachment
2020
include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
2121

22+
// Dynamic IP address assignment configuration with Whereabouts
23+
include::modules/nw-multus-whereabouts.adoc[leveloffset=+3]
24+
2225
// Configuring SR-IOV additional network
2326
include::modules/nw-sriov-network-attachment.adoc[leveloffset=+1]
2427

‎networking/multiple_networks/configuring-additional-network.adoc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,9 @@ include::modules/configuring-pods-static-ip.adoc[leveloffset=+3]
231231
// Configuration of IP address assignment for an additional network
232232
include::modules/nw-multus-ipam-object.adoc[leveloffset=+1]
233233

234+
// Dynamic IP address assignment configuration with Whereabouts
235+
include::modules/nw-multus-whereabouts.adoc[leveloffset=+2]
236+
234237
// Creating a whereabouts-reconciler daemon set
235238
include::modules/nw-multus-creating-whereabouts-reconciler-daemon-set.adoc[leveloffset=+2]
236239

0 commit comments

Comments
 (0)