Skip to content

Commit 0ded40e

Browse files
authored
Merge pull request #80997 from sr1kar99/1996-configuring-ipsec-for-mno
2 parents 7b25ecf + a33571c commit 0ded40e

File tree

4 files changed

+298
-105
lines changed

4 files changed

+298
-105
lines changed

edge_computing/ztp-deploying-far-edge-sites.adoc

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,12 +44,29 @@ include::modules/ztp-configuring-ipsec-using-ztp-and-siteconfig.adoc[leveloffset
4444
[role="_additional-resources"]
4545
.Additional resources
4646

47+
* xref:../edge_computing/ztp-deploying-far-edge-sites.adoc#ztp-verifying-ipsec_ztp-deploying-far-edge-sites[Verifying the IPsec encryption]
48+
4749
* xref:../networking/network_security/configuring-ipsec-ovn.adoc#configuring-ipsec-ovn[Configuring IPsec encryption]
4850
4951
* xref:../networking/network_security/configuring-ipsec-ovn.adoc#nw-ovn-ipsec-encryption_configuring-ipsec-ovn[Encryption protocol and IPsec mode]
5052
5153
* xref:../edge_computing/ztp-deploying-far-edge-sites.adoc#ztp-deploying-far-edge-sites[Installing managed clusters with {rh-rhacm} and SiteConfig resources]
5254
55+
include::modules/ztp-configuring-ipsec-using-ztp-and-siteconfig-for-mno.adoc[leveloffset=+2]
56+
57+
[role="_additional-resources"]
58+
.Additional resources
59+
60+
* xref:../edge_computing/ztp-deploying-far-edge-sites.adoc#ztp-verifying-ipsec_ztp-deploying-far-edge-sites[Verifying the IPsec encryption]
61+
62+
* xref:../networking/network_security/configuring-ipsec-ovn.adoc#configuring-ipsec-ovn[Configuring IPsec encryption]
63+
64+
* xref:../networking/network_security/configuring-ipsec-ovn.adoc#nw-ovn-ipsec-encryption_configuring-ipsec-ovn[Encryption protocol and IPsec mode]
65+
66+
* xref:../edge_computing/ztp-deploying-far-edge-sites.adoc#ztp-deploying-far-edge-sites[Installing managed clusters with {rh-rhacm} and SiteConfig resources]
67+
68+
include::modules/ztp-verifying-ipsec.adoc[leveloffset=+2]
69+
5370
include::modules/ztp-sno-siteconfig-config-reference.adoc[leveloffset=+2]
5471

5572
[role="_additional-resources"]
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * scalability_and_performance/ztp_far_edge/ztp-advanced-install-ztp.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="ztp-configuring-ipsec-using-ztp-and-siteconfig-for-mno_{context}"]
7+
= Configuring IPsec encryption for multi-node clusters using {ztp} and SiteConfig resources
8+
9+
You can enable IPsec encryption in managed multi-node clusters that you install using {ztp} and {rh-rhacm-first}.
10+
You can encrypt traffic between the managed cluster and IPsec endpoints external to the managed cluster. All network traffic between nodes on the OVN-Kubernetes cluster network is encrypted with IPsec in Transport mode.
11+
12+
.Prerequisites
13+
14+
* You have installed the OpenShift CLI (`oc`).
15+
16+
* You have logged in to the hub cluster as a user with `cluster-admin` privileges.
17+
18+
* You have configured {rh-rhacm} and the hub cluster for generating the required installation and policy custom resources (CRs) for managed clusters.
19+
20+
* You have created a Git repository where you manage your custom site configuration data.
21+
The repository must be accessible from the hub cluster and be defined as a source repository for the Argo CD application.
22+
23+
* You have installed the `butane` utility version 0.20.0 or later.
24+
25+
* You have a PKCS#12 certificate for the IPsec endpoint and a CA cert in PEM format.
26+
27+
* You have installed the NMState Operator.
28+
29+
.Procedure
30+
31+
. Extract the latest version of the `ztp-site-generate` container source and merge it with your repository where you manage your custom site configuration data.
32+
33+
. Configure the `optional-extra-manifest/ipsec/ipsec-config-policy.yaml` file with the required values that configure IPsec in the cluster.
34+
+
35+
.`ConfigurationPolicy` object for creating an IPsec configuration
36+
[source,yaml]
37+
----
38+
apiVersion: policy.open-cluster-management.io/v1
39+
kind: ConfigurationPolicy
40+
metadata:
41+
name: policy-config
42+
spec:
43+
namespaceSelector:
44+
include: ["default"]
45+
exclude: []
46+
matchExpressions: []
47+
matchLabels: {}
48+
remediationAction: inform
49+
severity: low
50+
evaluationInterval:
51+
compliant:
52+
noncompliant:
53+
object-templates-raw: |
54+
{{- range (lookup "v1" "Node" "" "").items }}
55+
- complianceType: musthave
56+
objectDefinition:
57+
kind: NodeNetworkConfigurationPolicy
58+
apiVersion: nmstate.io/v1
59+
metadata:
60+
name: {{ .metadata.name }}-ipsec-policy
61+
spec:
62+
nodeSelector:
63+
kubernetes.io/hostname: {{ .metadata.name }}
64+
desiredState:
65+
interfaces:
66+
- name: hosta_conn
67+
type: ipsec
68+
libreswan:
69+
left: '%defaultroute'
70+
leftid: '%fromcert'
71+
leftmodecfgclient: false
72+
leftcert: left_server <1>
73+
leftrsasigkey: '%cert'
74+
right: <external_host> <2>
75+
rightid: '%fromcert'
76+
rightrsasigkey: '%cert'
77+
rightsubnet: <external_address> <3>
78+
ikev2: insist <4>
79+
type: tunnel
80+
----
81+
<1> The value of this field must match with the name of the certificate used on the remote system.
82+
<2> Replace `<external_host>` with the external host IP address or DNS hostname.
83+
<3> Replace `<external_address>` with the IP subnet of the external host on the other side of the IPsec tunnel.
84+
<4> Use the IKEv2 VPN encryption protocol only. Do not use IKEv1, which is deprecated.
85+
86+
. Add the following certificates to the `optional-extra-manifest/ipsec` folder:
87+
88+
** `left_server.p12`: The certificate bundle for the IPsec endpoints
89+
90+
** `ca.pem`: The certificate authority that you signed your certificates with
91+
+
92+
The certificate files are required for the Network Security Services (NSS) database on each host. These files are imported as part of the Butane configuration in later steps.
93+
94+
. Open a shell prompt at the `optional-extra-manifest/ipsec` folder of the Git repository where you maintain your custom site configuration data.
95+
96+
. Run the `optional-extra-manifest/ipsec/import-certs.sh` script to generate the required Butane and `MachineConfig` CRs to import the external certs.
97+
+
98+
If the PKCS#12 certificate is protected with a password, set the `-W` argument.
99+
+
100+
.Example output
101+
[source,terminal]
102+
----
103+
out
104+
└── argocd
105+
└── example
106+
└── optional-extra-manifest
107+
└── ipsec
108+
├── 99-ipsec-master-import-certs.bu <1>
109+
├── 99-ipsec-master-import-certs.yaml <1>
110+
├── 99-ipsec-worker-import-certs.bu <1>
111+
├── 99-ipsec-worker-import-certs.yaml <1>
112+
├── import-certs.sh
113+
├── ca.pem <2>
114+
├── left_server.p12 <2>
115+
├── enable-ipsec.yaml
116+
├── ipsec-config-policy.yaml
117+
└── README.md
118+
----
119+
<1> The `ipsec/import-certs.sh` script generates the Butane and endpoint configuration CRs.
120+
<2> Add the `ca.pem` and `left_server.p12` certificate files that are relevant to your network.
121+
122+
. Create a `custom-manifest/` folder in the repository where you manage your custom site configuration data and add the `enable-ipsec.yaml` and `99-ipsec-*` YAML files to the directory.
123+
+
124+
.Example `siteconfig` directory
125+
[source,terminal]
126+
----
127+
siteconfig
128+
├── site1-mno-du.yaml
129+
├── extra-manifest/
130+
└── custom-manifest
131+
├── enable-ipsec.yaml
132+
├── 99-ipsec-master-import-certs.yaml
133+
└── 99-ipsec-worker-import-certs.yaml
134+
----
135+
136+
. In your `SiteConfig` CR, add the `custom-manifest/` directory to the `extraManifests.searchPaths` field, as in the following example:
137+
+
138+
[source,yaml]
139+
----
140+
clusters:
141+
- clusterName: "site1-mno-du"
142+
networkType: "OVNKubernetes"
143+
extraManifests:
144+
searchPaths:
145+
- extra-manifest/
146+
- custom-manifest/
147+
----
148+
149+
. Include the `ipsec-config-policy.yaml` config policy file in the `source-crs` directory in GitOps and reference the file in one of the `PolicyGenerator` CRs.
150+
151+
. Commit the `SiteConfig` CR changes and updated files in your Git repository and push the changes to provision the managed cluster and configure IPsec encryption.
152+
+
153+
The Argo CD pipeline detects the changes and begins the managed cluster deployment.
154+
+
155+
During cluster provisioning, the {ztp} pipeline appends the CRs in the `custom-manifest/` directory to the default set of extra manifests stored in the `extra-manifest/` directory.
156+
157+
.Verification
158+
159+
For information about verifying the IPsec encryption, see "Verifying the IPsec encryption".

modules/ztp-configuring-ipsec-using-ztp-and-siteconfig.adoc

Lines changed: 26 additions & 105 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,11 @@
77
= Configuring IPsec encryption for {sno} clusters using {ztp} and SiteConfig resources
88

99
You can enable IPsec encryption in managed {sno} clusters that you install using {ztp} and {rh-rhacm-first}.
10-
You can encrypt external traffic between pods and IPsec endpoints external to the managed cluster. All pod-to-pod network traffic between nodes on the OVN-Kubernetes cluster network is encrypted with IPsec in Transport mode.
10+
You can encrypt traffic between the managed cluster and IPsec endpoints external to the managed cluster. All network traffic between nodes on the OVN-Kubernetes cluster network is encrypted with IPsec in Transport mode.
1111

12-
[NOTE]
12+
[IMPORTANT]
1313
====
14-
In {product-title} {product-version}, deploying IPsec encryption by using {ztp} and {rh-rhacm} is validated for {sno} clusters only.
15-
16-
The {ztp} IPsec implementation assumes you are deploying to a resource constrained platform.
17-
As such, you install the feature with a single `MachineConfig` CR only, and you do not need to install the NMState Operator on the {sno} cluster as a prerequisite.
14+
You can also configure IPsec encryption for {sno} clusters with an additional worker node by following this procedure. It is recommended to use the `MachineConfig` custom resource (CR) to configure IPsec encryption for {sno} clusters and {sno} clusters with an additional worker node because of their low resource availability.
1815
====
1916

2017
.Prerequisites
@@ -28,7 +25,7 @@ As such, you install the feature with a single `MachineConfig` CR only, and you
2825
* You have created a Git repository where you manage your custom site configuration data.
2926
The repository must be accessible from the hub cluster and be defined as a source repository for the Argo CD application.
3027
31-
* You have installed the `butane` utility, version 0.20.0 or higher.
28+
* You have installed the `butane` utility version 0.20.0 or later.
3229
3330
* You have a PKCS#12 certificate for the IPsec endpoint and a CA cert in PEM format.
3431
@@ -44,35 +41,37 @@ interfaces:
4441
- name: hosta_conn
4542
type: ipsec
4643
libreswan:
47-
left: <cluster_node> <1>
44+
left: '%defaultroute'
4845
leftid: '%fromcert'
4946
leftmodecfgclient: false
50-
leftcert: <left_cert> <2>
47+
leftcert: left_server <1>
5148
leftrsasigkey: '%cert'
52-
right: <external_host> <3>
49+
right: <external_host> <2>
5350
rightid: '%fromcert'
5451
rightrsasigkey: '%cert'
55-
rightsubnet: <external_address> <4>
56-
ikev2: insist <5>
52+
rightsubnet: <external_address> <3>
53+
ikev2: insist <4>
5754
type: tunnel
5855
----
59-
<1> Replace `<cluster_node>` with the IP address or DNS hostname of the cluster node for the cluster-side IPsec tunnel.
60-
<2> Replace `<left_cert>` with the IPsec certificate nickname.
61-
<3> Replace `<external_host>` with the external host IP address or DNS hostname.
62-
<4> Replace `<external_address>` with the IP address or subnet of the external host on the other side of the IPsec tunnel.
63-
<5> Use the IKEv2 VPN encryption protocol only. Do not use IKEv1, which is deprecated.
56+
<1> The value of this field must match with the name of the certificate used on the remote system.
57+
<2> Replace `<external_host>` with the external host IP address or DNS hostname.
58+
<3> Replace `<external_address>` with the IP subnet of the external host on the other side of the IPsec tunnel.
59+
<4> Use the IKEv2 VPN encryption protocol only. Do not use IKEv1, which is deprecated.
6460

65-
. Add your `ca.pem` and `left_server.p12` certificates to the `optional-extra-manifest/ipsec` folder.
66-
The certificate files are required for the Network Security Services (NSS) database on each host. These files are imported as part of the Butane configuration in later steps.
61+
. Add the following certificates to the `optional-extra-manifest/ipsec` folder:
6762

68-
.. `left_server.p12`: The certificate bundle for the IPsec endpoints
63+
** `left_server.p12`: The certificate bundle for the IPsec endpoints
6964

70-
.. `ca.pem`: The certificate authority that you signed your certificates with
65+
** `ca.pem`: The certificate authority that you signed your certificates with
66+
+
67+
The certificate files are required for the Network Security Services (NSS) database on each host. These files are imported as part of the Butane configuration in later steps.
7168

7269
. Open a shell prompt at the `optional-extra-manifest/ipsec` folder of the Git repository where you maintain your custom site configuration data.
7370

7471
. Run the `optional-extra-manifest/ipsec/build.sh` script to generate the required Butane and `MachineConfig` CRs files.
7572
+
73+
If the PKCS#12 certificate is protected with a password, set the `-W` argument.
74+
+
7675
.Example output
7776
[source,terminal]
7877
----
@@ -82,12 +81,12 @@ out
8281
└── optional-extra-manifest
8382
└── ipsec
8483
├── 99-ipsec-master-endpoint-config.bu <1>
85-
├── 99-ipsec-master-endpoint-config.yaml
86-
├── 99-ipsec-worker-endpoint-config.bu
87-
├── 99-ipsec-worker-endpoint-config.yaml
84+
├── 99-ipsec-master-endpoint-config.yaml <1>
85+
├── 99-ipsec-worker-endpoint-config.bu <1>
86+
├── 99-ipsec-worker-endpoint-config.yaml <1>
8887
├── build.sh
8988
├── ca.pem <2>
90-
├── left_server.p12
89+
├── left_server.p12 <2>
9190
├── enable-ipsec.yaml
9291
├── ipsec-endpoint-config.yml
9392
└── README.md
@@ -128,86 +127,8 @@ clusters:
128127
+
129128
The Argo CD pipeline detects the changes and begins the managed cluster deployment.
130129
+
131-
During cluster provisioning, the {ztp} pipeline appends the CRs in the `/custom-manifest` directory to the default set of extra manifests stored in `extra-manifest/`.
130+
During cluster provisioning, the {ztp} pipeline appends the CRs in the `custom-manifest/` directory to the default set of extra manifests stored in the `extra-manifest/` directory.
132131

133132
.Verification
134133

135-
To verify that the IPsec encryption is successfully applied in the managed {sno} cluster, perform the following steps:
136-
137-
. Start a debug pod for the managed cluster by running the following command:
138-
+
139-
[source,terminal]
140-
----
141-
$ oc debug node/<node_name>
142-
----
143-
144-
. Check that the IPsec policy is applied in the cluster node:
145-
+
146-
[source,terminal]
147-
----
148-
sh-5.1# ip xfrm policy
149-
----
150-
+
151-
.Example output
152-
[source,terminal]
153-
----
154-
src 172.16.123.0/24 dst 10.1.232.10/32
155-
dir out priority 1757377 ptype main
156-
tmpl src 10.1.28.190 dst 10.1.232.10
157-
proto esp reqid 16393 mode tunnel
158-
src 10.1.232.10/32 dst 172.16.123.0/24
159-
dir fwd priority 1757377 ptype main
160-
tmpl src 10.1.232.10 dst 10.1.28.190
161-
proto esp reqid 16393 mode tunnel
162-
src 10.1.232.10/32 dst 172.16.123.0/24
163-
dir in priority 1757377 ptype main
164-
tmpl src 10.1.232.10 dst 10.1.28.190
165-
proto esp reqid 16393 mode tunnel
166-
----
167-
168-
. Check that the IPsec tunnel is up and connected:
169-
+
170-
[source,terminal]
171-
----
172-
sh-5.1# ip xfrm state
173-
----
174-
+
175-
.Example output
176-
[source,terminal]
177-
----
178-
src 10.1.232.10 dst 10.1.28.190
179-
proto esp spi 0xa62a05aa reqid 16393 mode tunnel
180-
replay-window 0 flag af-unspec esn
181-
auth-trunc hmac(sha1) 0x8c59f680c8ea1e667b665d8424e2ab749cec12dc 96
182-
enc cbc(aes) 0x2818a489fe84929c8ab72907e9ce2f0eac6f16f2258bd22240f4087e0326badb
183-
anti-replay esn context:
184-
seq-hi 0x0, seq 0x0, oseq-hi 0x0, oseq 0x0
185-
replay_window 128, bitmap-length 4
186-
00000000 00000000 00000000 00000000
187-
src 10.1.28.190 dst 10.1.232.10
188-
proto esp spi 0x8e96e9f9 reqid 16393 mode tunnel
189-
replay-window 0 flag af-unspec esn
190-
auth-trunc hmac(sha1) 0xd960ddc0a6baaccb343396a51295e08cfd8aaddd 96
191-
enc cbc(aes) 0x0273c02e05b4216d5e652de3fc9b3528fea94648bc2b88fa01139fdf0beb27ab
192-
anti-replay esn context:
193-
seq-hi 0x0, seq 0x0, oseq-hi 0x0, oseq 0x0
194-
replay_window 128, bitmap-length 4
195-
00000000 00000000 00000000 00000000
196-
----
197-
198-
. Ping a known IP in the external host subnet.
199-
For example, ping an IP in the `rightsubnet` range that you set in `ipsec/ipsec-endpoint-config.yaml`:
200-
+
201-
[source,terminal]
202-
----
203-
sh-5.1# ping 172.16.110.8
204-
----
205-
+
206-
.Example output
207-
[source,terminal]
208-
----
209-
sh-5.1# ping 172.16.110.8
210-
PING 172.16.110.8 (172.16.110.8) 56(84) bytes of data.
211-
64 bytes from 172.16.110.8: icmp_seq=1 ttl=64 time=153 ms
212-
64 bytes from 172.16.110.8: icmp_seq=2 ttl=64 time=155 ms
213-
----
134+
For information about verifying the IPsec encryption, see "Verifying the IPsec encryption".

0 commit comments

Comments
 (0)