Skip to content

Commit 655336f

Browse files
ShaunaDiazShaunaDiaz
committed
OSDOCS-9427: adds multus procedures MicroShift
1 parent 8d4c86f commit 655336f

9 files changed

+520
-4
lines changed

‎_topic_maps/_topic_map_ms.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,8 @@ Topics:
422422
Topics:
423423
- Name: About using multiple networks
424424
File: microshift-cni-multus
425+
- Name: Configuring and using multiple networks
426+
File: microshift-cni-multus-using
425427
- Name: Firewall configuration
426428
File: microshift-firewall
427429
- Name: Networking settings for fully disconnected hosts

‎microshift_networking/microshift_multiple_networks/microshift-cni-multus-using.adoc

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="microshift-cni-multus-using"]
3+
= Configuring and using multiple networks
4+
include::_attributes/attributes-microshift.adoc[]
5+
:context: microshift-cni-multus-using
6+
7+
toc::[]
8+
9+
After you have installed the {microshift-short} Multus Container Network Interface (CNI), you can use other networking plugins by using configurations.
10+
11+
include::modules/microshift-cni-multus-using-ipam.adoc[leveloffset=+1]
12+
13+
include::modules/microshift-cni-multus-nad-additional-network.adoc[leveloffset=+1]
14+
15+
//MicroShift-edited version of OCP procedure
16+
include::modules/microshift-nw-multus-add-pod.adoc[leveloffset=+1]
17+
18+
include::modules/microshift-cni-multus-add-network-example-config.adoc[leveloffset=+1]
19+
20+
//OCP procedure, edit with conditionals and care
21+
include::modules/nw-multus-remove-pod.adoc[leveloffset=+1]
22+
23+
include::modules/microshift-cni-multus-troubleshoot.adoc[leveloffset=+1]
24+
25+
[id="additional-resources_microshift-cni-multus-using_{context}"]
26+
== Additional resources
27+
* xref:../../microshift_networking/microshift_multiple_networks/microshift-cni-multus.adoc#microshift-cni-multus[About using multiple networks]
28+
29+
* link:https://access.redhat.com/documentation/en-us/openshift_container_platform/4.15/html/networking/multiple-networks#nw-multus-ipam-object_configuring-additional-network[Configuration of IP address assignment for an additional network]

‎microshift_networking/microshift_multiple_networks/microshift-cni-multus.adoc

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,14 @@ include::modules/microshift-install-multus-running-cluster.adoc[leveloffset=+1]
1414

1515
//OCP module, edit with conditionals and care
1616
include::modules/nw-multus-bridge-object.adoc[leveloffset=+1]
17-
//Q: do we need to remove VLAN from this reference for microshift?
1817

1918
//OCP module, edit with conditionals and care
2019
include::modules/nw-multus-ipvlan-object.adoc[leveloffset=+1]
2120

2221
//OCP module, edit with conditionals and care
2322
include::modules/nw-multus-macvlan-object.adoc[leveloffset=+1]
2423

25-
[role="_additional-resources"]
26-
.Additional resources
24+
[id="additional-resources_microshift-cni-multus_{context}"]
25+
== Additional resources
2726

28-
//* add cross refs to procedures
27+
* xref:../../microshift_networking/microshift_multiple_networks/microshift-cni-multus-using.adoc#microshift-cni-multus-using[Configuring and using multiple networks]

‎modules/microshift-cni-multus-add-network-example-config.adoc

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * microshift_networking/microshift_multiple_networks/microshift-cni-multus-using.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="microshift-cni-multus-add-network-example-config_{context}"]
7+
= Configuring an additional network
8+
9+
After you have created the NetworkAttachmentDefinition object and applied it, use the following example procedure to configure an additional network. In this example, the `bridge` type additional network is used. You can also use this workflow for other additional network types.
10+
11+
.Prerequisite
12+
. You created and applied the `NetworkAttachmentDefinition` object configuration.
13+
14+
.Procedure
15+
. Verify that the bridge was created on the host by running the following command:
16+
+
17+
[source,terminal]
18+
----
19+
$ ip a show br-test
20+
----
21+
+
22+
.Example output
23+
[source,terminal]
24+
----
25+
22: br-test: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
26+
link/ether 96:bf:ca:be:1d:15 brd ff:ff:ff:ff:ff:ff
27+
inet6 fe80::34e2:bbff:fed2:31f2/64 scope link
28+
valid_lft forever preferred_lft forever
29+
----
30+
31+
. Configure an IP address for the bridge by running the following command:
32+
+
33+
[source,terminal]
34+
----
35+
$ sudo ip addr add 10.10.0.10/24 dev br-test
36+
----
37+
38+
. Verify that the IP address configuration is added to the bridge by running the following command:
39+
+
40+
[source,terminal]
41+
----
42+
$ ip a show br-test
43+
----
44+
+
45+
.Example output
46+
[source,terminal]
47+
----
48+
22: br-test: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
49+
link/ether 96:bf:ca:be:1d:15 brd ff:ff:ff:ff:ff:ff
50+
inet 10.10.0.10/24 scope global br-test <1>
51+
valid_lft forever preferred_lft forever
52+
inet6 fe80::34e2:bbff:fed2:31f2/64 scope link
53+
valid_lft forever preferred_lft forever
54+
----
55+
<1> The IP address is configured as expected.
56+
57+
. Verify the IP address of the pod by running the following command:
58+
+
59+
[source,terminal]
60+
----
61+
$ oc get pod test-bridge --output=jsonpath='{.metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status}'
62+
----
63+
+
64+
.Example output
65+
[source,terminal]
66+
----
67+
[{
68+
"name": "ovn-kubernetes",
69+
"interface": "eth0",
70+
"ips": [
71+
"10.42.0.17"
72+
],
73+
"mac": "0a:58:0a:2a:00:11",
74+
"default": true,
75+
"dns": {}
76+
},{
77+
"name": "default/bridge-conf", <1>
78+
"interface": "net1",
79+
"ips": [
80+
"10.10.0.20"
81+
],
82+
"mac": "82:01:98:e5:0c:b7",
83+
"dns": {}
84+
----
85+
<1> The bridge additional network is attached as expected.
86+
87+
. Optional: You can use `oc exec` to access the pod and confirm its interfaces by using the `ip` command:
88+
+
89+
[source,terminal]
90+
----
91+
$ oc exec -ti test-bridge -- ip a
92+
----
93+
+
94+
.Example output
95+
[source,terminal]
96+
----
97+
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
98+
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
99+
inet 127.0.0.1/8 scope host lo
100+
valid_lft forever preferred_lft forever
101+
inet6 ::1/128 scope host
102+
valid_lft forever preferred_lft forever
103+
2: eth0@if21: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
104+
link/ether 0a:58:0a:2a:00:11 brd ff:ff:ff:ff:ff:ff
105+
inet 10.42.0.17/24 brd 10.42.0.255 scope global eth0
106+
valid_lft forever preferred_lft forever
107+
inet6 fe80::858:aff:fe2a:11/64 scope link
108+
valid_lft forever preferred_lft forever
109+
3: net1@if23: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
110+
link/ether 82:01:98:e5:0c:b7 brd ff:ff:ff:ff:ff:ff
111+
inet 10.10.0.20/24 brd 10.10.0.255 scope global net1 <1>
112+
valid_lft forever preferred_lft forever
113+
inet6 fe80::8001:98ff:fee5:cb7/64 scope link
114+
valid_lft forever preferred_lft forever
115+
----
116+
<1> Pod is attached to the 10.10.0.20 IP address on the `net1 interface` as expected.
117+
118+
. Confirm that the connection is working as expected by accessing the HTTP server in the pod from the {microshift-short} host. Use the following command:
119+
+
120+
[source,terminal]
121+
----
122+
$ curl 10.10.0.20:8080
123+
----
124+
+
125+
.Example output
126+
[source,terminal]
127+
----
128+
Hello MicroShift
129+
----

‎modules/microshift-cni-multus-nad-additional-network.adoc

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * microshift_networking/microshift_multiple_networks/microshift-cni-multus-using.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="microshift-cni-multus-nad-additional-network_{context}"]
7+
= Creating a NetworkAttachmentDefinition for an additional network
8+
9+
Use the following procedure to create a `NetworkAttachmentDefinition` configuration file for an additional network. In this example, a bridge-type interface is used. You can also use the example workflow here that uses `host-local` IP address management (IPAM) to configure other supported additional network types.
10+
11+
[IMPORTANT]
12+
====
13+
If you use `bridge` and the `dhcp` IPAM, a DHCP server listening on the bridged network is required. If you are also using a firewall, configuring the firewalld service to allow DHCP traffic on the network zone is also required. You can run the `firewall-cmd --remove-service=dhcp` command in this case.
14+
====
15+
16+
.Prerequisites
17+
18+
* The {microshift-short} Multus CNI is installed.
19+
* The OpenShift CLI (`oc`) is installed.
20+
* The cluster is running.
21+
22+
.Procedure
23+
24+
. Optional: Verify that the {microshift-short} cluster is running with the Multus CNI by running the following command:
25+
+
26+
[source,terminal]
27+
----
28+
$ oc get pods -n openshift-multus
29+
----
30+
+
31+
.Example output
32+
[source,terminal]
33+
----
34+
NAME READY STATUS RESTARTS AGE
35+
dhcp-daemon-dfbzw 1/1 Running 0 5h
36+
multus-rz8xc 1/1 Running 0 5h
37+
----
38+
39+
. Create a `NetworkAttachmentDefinition` configuration file by running the following command and using the following example file for reference:
40+
+
41+
[source,terminal]
42+
----
43+
$ oc apply -f network-attachment-definition.yaml
44+
----
45+
+
46+
.Example `NetworkAttachmentDefinition` file
47+
[source,yaml]
48+
----
49+
apiVersion: "k8s.cni.cncf.io/v1"
50+
kind: NetworkAttachmentDefinition
51+
metadata:
52+
name: bridge-conf
53+
spec:
54+
config: '{
55+
"cniVersion": "0.4.0",
56+
"type": "bridge", <1>
57+
"bridge": "br-test", <2>
58+
"mode": "bridge",
59+
"ipam": {
60+
"type": "host-local", <3>
61+
"ranges": [
62+
[
63+
{
64+
"subnet": "10.10.0.0/24",
65+
"rangeStart": "10.10.0.20",
66+
"rangeEnd": "10.10.0.50",
67+
"gateway": "10.10.0.254"
68+
}
69+
],
70+
[
71+
{
72+
"subnet": "fd00:IJKL:MNOP:10::0/64", <4>
73+
"rangeStart": "fd00:IJKL:MNOP:10::1",
74+
"rangeEnd": "fd00:IJKL:MNOP:10::9"
75+
"dataDir": "/var/lib/cni/br-test"
76+
}
77+
}'
78+
----
79+
<1> The `type` value specifies a name of the CNI plugin. This example uses the `bridge` type.
80+
<2> The `bridge` value is name of the bridge on the {microshift-short} host that is used. The additional interface of the pod is connected to that bridge. If the interface does not exist on the host, the Bridge CNI creates it. If the interface already exists, it is reused. In this example, the name of the interface is `br-test`.
81+
<3> The IPAM type.
82+
<4> IPv6 addresses can be added to the secondary interface.
83+
+
84+
--
85+
[NOTE]
86+
====
87+
Using the name of the bridge is specific to the `bridge` type of plugin. Other plugins use different fields in their `NetworkAttachmentDefinitions`. For example, the `macvlan` and `ipvlan` configurations use `master` to specify the host interface to attach.
88+
====
89+
--

‎modules/microshift-cni-multus-troubleshoot.adoc

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * microshift_networking/microshift_multiple_networks/microshift-cni-multus-using.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="microshift-cni-multus-troubleshoot_{context}"]
7+
= Troubleshooting Multus networking
8+
9+
If the settings for multiple networks are not configured properly, pods can fail to start. The following steps can help you solve for a couple common scenarios.
10+
11+
[id="Pod-networking-cannot-be-configured_{context}"]
12+
== Pod networking cannot be configured
13+
14+
If the Multus CNI plugin cannot apply networking annotations to a pod, the pod does not start. Pods can also fail to start if any of the additional network CNIs fail.
15+
16+
.Example error
17+
[source,terminal]
18+
----
19+
Warning NoNetworkFound 0s multus cannot find a network-attachment-definitio (asdasd) in namespace (default): network-attachment-definitions.k8s.cni.cncf.io "bad-ref-doesnt-exist" not found
20+
----
21+
22+
In this case, you can take the following steps to trouble CNI failures:
23+
24+
* Verify the values in both the `NetworkAttachmentDefinitions` and the annotations.
25+
* Remove the annotation to verify whether the pod is created successfully with just the default network. If not, this might indicate a networking problem other than the Multus configuration.
26+
* If you are a device administrator, you can inspect the `crio.service` or `microshift.service` logs, paying special attention to those that are generated by the `kubelet`.
27+
+
28+
For example, the following error from the `kubelet` shows that the primary CNI is not running. This situation can be caused by pods not starting or because of a CRI-O misconfiguration such as an incorrect `cni_default_network` setting.
29+
+
30+
.Example kubelet-generated error
31+
[source,terminal]
32+
----
33+
Feb 06 13:47:31 dev microshift[1494]: kubelet E0206 13:47:31.163290 1494 pod_workers.go:1298] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/cni/net.d/. Has your network provider started?" pod="default/samplepod" podUID="fe0f7f7a-8c47-4488-952b-8abc0d8e2602"
34+
----
35+
36+
[id="missing-nad_{context}"]
37+
== Missing configuration file
38+
39+
Sometimes a pod cannot be created because the annotations reference a `NetworkAttachmentDefinition` configuration YAML that does not exist. In this case an error such as the following is usually produced:
40+
41+
.Example log
42+
[source,terminal]
43+
----
44+
cannot find a network-attachment-definition (bad-conf) in namespace (default): network-attachment-definitions.k8s.cni.cncf.io "bad-conf" not found" pod="default/samplepod"`
45+
----
46+
.Example error output
47+
[source,terminal]
48+
----
49+
"CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_samplepod_default_5fa13105-1bfb-4c6b-aee7-3437cfb50e25_0(7517818bd8e85f07b551f749c7529be88b4e7daef0dd572d049aa636950c76c6): error adding pod default_samplepod to CNI network \"multus-cni-network\": plugin type=\"multus\" name=\"multus-cni-network\" failed (add): Multus: [default/samplepod/5fa13105-1bfb-4c6b-aee7-3437cfb50e25]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in getting k8s network for pod: GetNetworkDelegates: failed getting the delegate: getKubernetesDelegate: cannot find a network-attachment-definition (bad-conf) in namespace (default): network-attachment-definitions.k8s.cni.cncf.io \"bad-conf\" not found" pod="default/samplepod"
50+
----
51+
52+
To fix this error, create and apply the `NetworkAttachmentDefinitions` YAML.

‎modules/microshift-cni-multus-using-ipam.adoc

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * microshift_networking/microshift_multiple_networks/microshift-cni-multus-using.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="IP-address-management-types-and-additional-networks_{context}"]
7+
= IP address management types and additional networks
8+
9+
IP addresses are provisioned for an additional network through an IP Address Management (IPAM) CNI plugin that you configure. Supported IP address provisioning types in {microshift-short} are `host-local`, `static`, and `dhcp`.
10+
11+
[id="bridge-interface-specifics_{context}"]
12+
== bridge interface specifics
13+
When using the `bridge` type interface and the `dhcp` IPAM, a DHCP server listening on the bridged network is required. If you are using a firewall, configuring the `firewalld` service by running the `firewall-cmd --remove-service=dhcp` command to allow DHCP traffic on the network zone is also required.
14+
15+
[id="macvlan-interface-specifics_{context}"]
16+
== macvlan interface specifics
17+
The `macvlan` type interface accesses the network that the host is connected to. This means that the interface can receive an IP address from the DHCP server on the host network if the `dhcp` IPAM plugin is used.
18+
19+
[id="ipvlan-interface-specifics_{context}"]
20+
== ipvlan interface specifics
21+
The `ipvlan` interface also has direct access to the host network, but shares a MAC address with the host interface. The `ipvlan` type interface cannot be used with the `dhcp` plugin because of the shared MAC address. The IPAM plugin does not support the DHCP protocol with `ClientID`.

0 commit comments

Comments
 (0)