Skip to content

Commit 3dac927

Browse files
dfitzmaudfitzmau
authored
Merge pull request #87205 from dfitzmau/OCPBUGS-16286-exIP
OCPBUGS-16286-exIP: Added more concept info about ExternelIP
2 parents 2da1931 + 0640f90 commit 3dac927

File tree

5 files changed

+178
-155
lines changed

5 files changed

+178
-155
lines changed

‎modules/configuration-externalip.adoc

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/configuring_ingress_cluster_traffic/configuring-externalip.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="configuration-externalip_{context}"]
7+
= Configuration for ExternalIP
8+
9+
Use of an external IP address in {product-title} is governed by the following parameters in the `Network.config.openshift.io` custom resource (CR) that is named `cluster`:
10+
11+
* `spec.externalIP.autoAssignCIDRs` defines an IP address block used by the load balancer when choosing an external IP address for the service. {product-title} supports only a single IP address block for automatic assignment. This configuration requires less steps than manually assigning ExternalIPs to services, which requires managing the port space of a limited number of shared IP addresses. If you enable automatic assignment, a `Service` object with `spec.type=LoadBalancer` is allocated an external IP address.
12+
13+
* `spec.externalIP.policy` defines the permissible IP address blocks when manually specifying an IP address. {product-title} does not apply policy rules to IP address blocks that you defined in the `spec.externalIP.autoAssignCIDRs` parameter.
14+
15+
If routed correctly, external traffic from the configured external IP address block can reach service endpoints through any TCP or UDP port that the service exposes.
16+
17+
[IMPORTANT]
18+
====
19+
As a cluster administrator, you must configure routing to externalIPs. You must also ensure that the IP address block you assign terminates at one or more nodes in your cluster. For more information, see link:https://kubernetes.io/docs/concepts/services-networking/service/#external-ips[Kubernetes External IPs].
20+
====
21+
22+
{product-title} supports both the automatic and manual assignment of IP addresses, where each address is guaranteed to be assigned to a maximum of one service. This configuration ensures that each service can expose its chosen ports regardless of the ports exposed by other services.
23+
24+
[NOTE]
25+
====
26+
To use IP address blocks defined by `autoAssignCIDRs` in {product-title}, you must configure the necessary IP address assignment and routing for your host network.
27+
====
28+
29+
The following YAML describes a service with an external IP address configured:
30+
31+
.Example `Service` object with `spec.externalIPs[]` set
32+
[source,yaml]
33+
----
34+
apiVersion: v1
35+
kind: Service
36+
metadata:
37+
name: http-service
38+
spec:
39+
clusterIP: 172.30.163.110
40+
externalIPs:
41+
- 192.168.132.253
42+
externalTrafficPolicy: Cluster
43+
ports:
44+
- name: highport
45+
nodePort: 31903
46+
port: 30102
47+
protocol: TCP
48+
targetPort: 30102
49+
selector:
50+
app: web
51+
sessionAffinity: None
52+
type: LoadBalancer
53+
status:
54+
loadBalancer:
55+
ingress:
56+
- ip: 192.168.132.253
57+
# ...
58+
----

‎modules/example-policy-objects.adoc

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/configuring_ingress_cluster_traffic/configuring-externalip.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="example-policy-objects_{context}"]
7+
= Example policy objects
8+
9+
The examples in this section show different `spec.externalIP.policy` configurations.
10+
11+
- In the following example, the policy prevents {product-title} from creating any service with a specified external IP address.
12+
+
13+
.Example policy to reject any value specified for `Service` object `spec.externalIPs[]`
14+
[source,yaml]
15+
----
16+
apiVersion: config.openshift.io/v1
17+
kind: Network
18+
metadata:
19+
name: cluster
20+
spec:
21+
externalIP:
22+
policy: {}
23+
# ...
24+
----
25+
26+
- In the following example, both the `allowedCIDRs` and `rejectedCIDRs` fields are set.
27+
+
28+
.Example policy that includes both allowed and rejected CIDR blocks
29+
[source,yaml]
30+
----
31+
apiVersion: config.openshift.io/v1
32+
kind: Network
33+
metadata:
34+
name: cluster
35+
spec:
36+
externalIP:
37+
policy:
38+
allowedCIDRs:
39+
- 172.16.66.10/23
40+
rejectedCIDRs:
41+
- 172.16.66.10/24
42+
# ...
43+
----
44+
45+
- In the following example, `policy` is set to `{}`. With this configuration, using the `oc get networks.config.openshift.io -o yaml` command to view the configuration means `policy` parameter does not show on the command output. The same behavior exists for `policy: null`.
46+
+
47+
.Example policy to allow any value specified for `Service` object `spec.externalIPs[]`
48+
[source,yaml]
49+
----
50+
apiVersion: config.openshift.io/v1
51+
kind: Network
52+
metadata:
53+
name: cluster
54+
spec:
55+
clusterNetwork:
56+
- cidr: 10.128.0.0/14
57+
hostPrefix: 23
58+
externalIP:
59+
policy: {}
60+
# ...
61+
----

‎modules/nw-externalip-about.adoc

Lines changed: 12 additions & 155 deletions
Original file line numberDiff line numberDiff line change
@@ -6,174 +6,31 @@
66
[id="nw-externalip-about_{context}"]
77
= About ExternalIP
88

9-
For non-cloud environments, {product-title} supports the assignment of external IP addresses to a `Service` object `spec.externalIPs[]` field through the *ExternalIP* facility.
10-
By setting this field, {product-title} assigns an additional virtual IP address to the service. The IP address can be outside the service network defined for the cluster.
11-
A service configured with an ExternalIP functions similarly to a service with `type=NodePort`, allowing you to direct traffic to a local node for load balancing.
9+
For non-cloud environments, {product-title} supports the use of the ExternalIP facility to specify external IP addresses in the `spec.externalIPs[]` parameter of the `Service` object. A service configured with an ExternalIP functions similarly to a service with `type=NodePort`, whereby you traffic directs to a local node for load balancing.
1210

13-
You must configure your networking infrastructure to ensure that the external IP address blocks that you define are routed to the cluster. As a result, the IP address is not configured in the network interfaces from nodes. To handle the traffic, you must configure the routing and access to the external IP by using a method such as static Address Resolution Protocol (ARP) entries.
14-
15-
{product-title} extends the ExternalIP functionality in Kubernetes by adding the following capabilities:
16-
17-
- Restrictions on the use of external IP addresses by users through a configurable policy
18-
- Allocation of an external IP address automatically to a service upon request
19-
20-
[WARNING]
11+
[IMPORTANT]
2112
====
22-
Disabled by default, use of ExternalIP functionality can be a security risk, because in-cluster traffic to an external IP address is directed to that service.
23-
This could allow cluster users to intercept sensitive traffic destined for external resources.
13+
For cloud environments, use the load balancer services for automatic deployment of a cloud load balancer to target the endpoints of a service.
2414
====
2515

26-
[IMPORTANT]
16+
After you specify a value for the parameter, {product-title} assigns an additional virtual IP address to the service. The IP address can exist outside of the service network that you defined for your cluster.
17+
18+
[WARNING]
2719
====
28-
This feature is supported only in non-cloud deployments.
29-
For cloud deployments, use the load balancer services for automatic deployment of a cloud load balancer to target the endpoints of a service.
20+
Because ExternalIP is disabled by default, enabling the ExternalIP functionality might introduce security risks for the service, because in-cluster traffic to an external IP address is directed to that service. This configuration means that cluster users could intercept sensitive traffic destined for external resources.
3021
====
3122

3223
You can use either a MetalLB implementation or an IP failover deployment to attach an ExternalIP resource to a service in the following ways:
3324

3425
Automatic assignment of an external IP::
35-
{product-title} automatically assigns an IP address from the `autoAssignCIDRs` CIDR block to the `spec.externalIPs[]` array when you create a `Service` object with `spec.type=LoadBalancer` set.
36-
In this case, {product-title} implements a non-cloud version of the load balancer service type and assigns IP addresses to the services.
37-
Automatic assignment is disabled by default and must be configured by a cluster administrator as described in the following section.
26+
{product-title} automatically assigns an IP address from the `autoAssignCIDRs` CIDR block to the `spec.externalIPs[]` array when you create a `Service` object with `spec.type=LoadBalancer` set. For this configuration, {product-title} implements a cloud version of the load balancer service type and assigns IP addresses to the services. Automatic assignment is disabled by default and must be configured by a cluster administrator as described in the "Configuration for ExternalIP" section.
3827

3928
Manual assignment of an external IP::
4029
{product-title} uses the IP addresses assigned to the `spec.externalIPs[]` array when you create a `Service` object. You cannot specify an IP address that is already in use by another service.
4130

42-
[id="configuration-externalip_{context}"]
43-
== Configuration for ExternalIP
44-
45-
Use of an external IP address in {product-title} is governed by the following fields in the `Network.config.openshift.io` CR named `cluster`:
46-
47-
* `spec.externalIP.autoAssignCIDRs` defines an IP address block used by the load balancer when choosing an external IP address for the service. {product-title} supports only a single IP address block for automatic assignment. This can be simpler than having to manage the port space of a limited number of shared IP addresses when manually assigning ExternalIPs to services. If automatic assignment is enabled, a `Service` object with `spec.type=LoadBalancer` is allocated an external IP address.
48-
* `spec.externalIP.policy` defines the permissible IP address blocks when manually specifying an IP address. {product-title} does not apply policy rules to IP address blocks defined by `spec.externalIP.autoAssignCIDRs`.
49-
50-
If routed correctly, external traffic from the configured external IP address block can reach service endpoints through any TCP or UDP port that the service exposes.
51-
52-
[IMPORTANT]
53-
====
54-
As a cluster administrator, you must configure routing to externalIPs. You must also ensure that the IP address block you assign terminates at one or more nodes in your cluster. For more information, see link:https://kubernetes.io/docs/concepts/services-networking/service/#external-ips[*Kubernetes External IPs*].
55-
====
56-
57-
{product-title} supports both the automatic and manual assignment of IP
58-
addresses, and each address is guaranteed to be assigned to a maximum of one
59-
service. This ensures that each service can expose its chosen ports regardless
60-
of the ports exposed by other services.
31+
After using either the MetalLB implementation or an IP failover deployment to host external IP address blocks, you must configure your networking infrastructure to ensure that the external IP address blocks are routed to your cluster. This configuration means that the IP address is not configured in the network interfaces from nodes. To handle the traffic, you must configure the routing and access to the external IP by using a method, such as static Address Resolution Protocol (ARP) entries.
6132

62-
[NOTE]
63-
====
64-
To use IP address blocks defined by `autoAssignCIDRs` in {product-title}, you must configure the necessary IP address assignment and routing for your host network.
65-
====
66-
67-
The following YAML describes a service with an external IP address configured:
68-
69-
.Example `Service` object with `spec.externalIPs[]` set
70-
[source,yaml]
71-
----
72-
apiVersion: v1
73-
kind: Service
74-
metadata:
75-
name: http-service
76-
spec:
77-
clusterIP: 172.30.163.110
78-
externalIPs:
79-
- 192.168.132.253
80-
externalTrafficPolicy: Cluster
81-
ports:
82-
- name: highport
83-
nodePort: 31903
84-
port: 30102
85-
protocol: TCP
86-
targetPort: 30102
87-
selector:
88-
app: web
89-
sessionAffinity: None
90-
type: LoadBalancer
91-
status:
92-
loadBalancer:
93-
ingress:
94-
- ip: 192.168.132.253
95-
----
96-
97-
[id="restrictions-on-ip-assignment_{context}"]
98-
== Restrictions on the assignment of an external IP address
99-
100-
As a cluster administrator, you can specify IP address blocks to allow and to reject.
101-
102-
Restrictions apply only to users without `cluster-admin` privileges. A cluster administrator can always set the service `spec.externalIPs[]` field to any IP address.
103-
104-
You configure IP address policy with a `policy` object defined by specifying the `spec.ExternalIP.policy` field.
105-
The policy object has the following shape:
106-
107-
[source,json]
108-
----
109-
{
110-
"policy": {
111-
"allowedCIDRs": [],
112-
"rejectedCIDRs": []
113-
}
114-
}
115-
----
116-
117-
When configuring policy restrictions, the following rules apply:
118-
119-
- If `policy={}` is set, then creating a `Service` object with `spec.ExternalIPs[]` set will fail. This is the default for {product-title}. The behavior when `policy=null` is set is identical.
120-
- If `policy` is set and either `policy.allowedCIDRs[]` or `policy.rejectedCIDRs[]` is set, the following rules apply:
121-
122-
* If `allowedCIDRs[]` and `rejectedCIDRs[]` are both set, then `rejectedCIDRs[]` has precedence over `allowedCIDRs[]`.
123-
* If `allowedCIDRs[]` is set, creating a `Service` object with `spec.ExternalIPs[]` will succeed only if the specified IP addresses are allowed.
124-
* If `rejectedCIDRs[]` is set, creating a `Service` object with `spec.ExternalIPs[]` will succeed only if the specified IP addresses are not rejected.
125-
126-
[id="example-policy-objects_{context}"]
127-
== Example policy objects
128-
129-
The examples that follow demonstrate several different policy configurations.
130-
131-
- In the following example, the policy prevents {product-title} from creating any service with an external IP address specified:
132-
+
133-
.Example policy to reject any value specified for `Service` object `spec.externalIPs[]`
134-
[source,yaml]
135-
----
136-
apiVersion: config.openshift.io/v1
137-
kind: Network
138-
metadata:
139-
name: cluster
140-
spec:
141-
externalIP:
142-
policy: {}
143-
...
144-
----
145-
146-
- In the following example, both the `allowedCIDRs` and `rejectedCIDRs` fields are set.
147-
+
148-
.Example policy that includes both allowed and rejected CIDR blocks
149-
[source,yaml]
150-
----
151-
apiVersion: config.openshift.io/v1
152-
kind: Network
153-
metadata:
154-
name: cluster
155-
spec:
156-
externalIP:
157-
policy:
158-
allowedCIDRs:
159-
- 172.16.66.10/23
160-
rejectedCIDRs:
161-
- 172.16.66.10/24
162-
...
163-
----
33+
{product-title} extends the ExternalIP functionality in Kubernetes by adding the following capabilities:
16434

165-
- In the following example, `policy` is set to `null`.
166-
If set to `null`, when inspecting the configuration object by entering `oc get networks.config.openshift.io -o yaml`, the `policy` field will not appear in the output.
167-
+
168-
.Example policy to allow any value specified for `Service` object `spec.externalIPs[]`
169-
[source,yaml]
170-
----
171-
apiVersion: config.openshift.io/v1
172-
kind: Network
173-
metadata:
174-
name: cluster
175-
spec:
176-
externalIP:
177-
policy: null
178-
...
179-
----
35+
- Restrictions on the use of external IP addresses by users through a configurable policy
36+
- Allocation of an external IP address automatically to a service upon request

‎modules/restrictions-on-ip-assignment.adoc

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/configuring_ingress_cluster_traffic/configuring-externalip.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="restrictions-on-ip-assignment_{context}"]
7+
= Restrictions on the assignment of an external IP address
8+
9+
As a cluster administrator, you can specify IP address blocks to allow and to reject IP addresses for a service. Restrictions apply only to users without `cluster-admin` privileges. A cluster administrator can always set the service `spec.externalIPs[]` field to any IP address.
10+
11+
You configure an IP address policy by specifying Classless Inter-Domain Routing (CIDR) address blocks for the `spec.ExternalIP.policy` parameter in the `policy` object.
12+
13+
.Example in JSON form of a `policy` object and its CIDR parameters
14+
[source,json]
15+
----
16+
{
17+
"policy": {
18+
"allowedCIDRs": [],
19+
"rejectedCIDRs": []
20+
}
21+
}
22+
----
23+
24+
When configuring policy restrictions, the following rules apply:
25+
26+
- If `policy` is set to `{}`, creating a `Service` object with `spec.ExternalIPs[]` results in a failed service. This setting is the default for {product-title}. The same behavior exists for `policy: null`.
27+
- If `policy` is set and either `policy.allowedCIDRs[]` or `policy.rejectedCIDRs[]` is set, the following rules apply:
28+
29+
* If `allowedCIDRs[]` and `rejectedCIDRs[]` are both set, `rejectedCIDRs[]` has precedence over `allowedCIDRs[]`.
30+
* If `allowedCIDRs[]` is set, creating a `Service` object with `spec.ExternalIPs[]` succeeds only if the specified IP addresses are allowed.
31+
* If `rejectedCIDRs[]` is set, creating a `Service` object with `spec.ExternalIPs[]` succeeds only if the specified IP addresses are not rejected.

‎networking/configuring_ingress_cluster_traffic/configuring-externalip.adoc

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,22 @@ This functionality is generally most useful for clusters installed on bare-metal
1717
// About ExternalIP
1818
include::modules/nw-externalip-about.adoc[leveloffset=+1]
1919

20+
[id="additional-resources_{context}"]
21+
== Additional resources
22+
23+
* xref:../../networking/configuring-ipfailover.adoc#configuring-ipfailover[Configuring IP failover]
24+
25+
* xref:../../networking/networking_operators/metallb-operator/about-metallb.adoc#about-metallb[About MetalLB and the MetalLB Operator]
26+
27+
// Configuration for ExternalIP
28+
include::modules/configuration-externalip.adoc[leveloffset=+1]
29+
30+
// Restrictions on the assignment of an external IP address
31+
include::modules/restrictions-on-ip-assignment.adoc[leveloffset=+1]
32+
33+
// Example policy objects
34+
include::modules/example-policy-objects.adoc[leveloffset=+1]
35+
2036
// ExternalIP address block configuration
2137
include::modules/nw-externalip-object.adoc[leveloffset=+1]
2238

0 commit comments

Comments
 (0)