Skip to content

Adds egress netpol docs #92159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

stevsmit
Copy link
Member

@stevsmit stevsmit commented Apr 14, 2025

@openshift-ci openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 14, 2025
@openshift-ci openshift-ci bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 16, 2025
@stevsmit stevsmit force-pushed the OCPBUGS-54674 branch 3 times, most recently from b1988cc to 912c8df Compare April 17, 2025 19:50

.Procedure

. Create the following `allow-n1-a-to-n2-b` network policy to allow pods across namespaces to communicate with each other. With this YAML, pods in the `project-a` namespace can communicate with pods in the `project-b` namespace, so long as those namespaces are labeled `networking/namespace: n1` and `networking/namespace: n2`, respectively. Save the YAML in the `allow-n1-a-to-n2-b` file:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] RedHat.TermsErrors: Use 'if' or 'when' rather than 'so long as'. For more information, see RedHat.TermsErrors.

@openshift-ci openshift-ci bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 18, 2025
Copy link

openshift-ci bot commented Apr 18, 2025

@stevsmit: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@bergerhoffer
Copy link
Contributor

The branch/enterprise-4.20 label has been added to this PR.

This is because your PR targets the main branch and is labeled for enterprise-4.19. And any PR going into main must also target the latest version branch (enterprise-4.20).

If the update in your PR does NOT apply to version 4.20 onward, please re-target this PR to go directly into the appropriate version branch or branches (enterprise-4.x) instead of main.

ingress:
- {}
----
<1> Apply this label to pods to enable the pod to receive traffic from outside sources.
Copy link

@asood-rh asood-rh Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<1> Apply this label to pods to enable the pod to receive traffic from outside sources.
<1> Apply this label to pods to enable the pods to receive traffic from outside sources.

+
[source,terminal]
----
$ oc exec -it test-pod-b -n project-b -- ping 10.132.0.44

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc exec -it test-pod-b -n project-b -- ping 10.132.0.44
$ oc exec -it test-pod-b -n project-b -- ping -c 2 10.132.0.44

You could just say send 2 ICMP packets

+
[source,terminal]
----
$ oc exec -it busybox-pod-a -n project-a -- ping 10.132.0.40

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc exec -it busybox-pod-a -n project-a -- ping 10.132.0.40
$ oc exec -it busybox-pod-a -n project-a -- ping -c 2 10.132.0.40

This would ensure user does not have to issue Ctl C to stop sending ICMP packets


.Prerequisites

* You have created the `deny-by-default` network policy and applied it to the necessary namespaces.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* You have created the `deny-by-default` network policy and applied it to the necessary namespaces.
* You have created the `deny-by-default` network policy and applied it to the necessary namespaces. The policy denies ingress traffic to pods in the project.


.Prerequisites

* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.
* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.

We do not have such a mode in OVNK clusters.

[id="nw-networkpolicy-allow-ingress_{context}"]
= Creating an allow ingress access network policy

With the `deny-by-default` network policy in place, no pods can talk to each other or receive traffic from external sources. One option to enable communication is to allow some pods to receive traffic. To do so, you can create the following `ingress-access` network policy. With this network policy, pods with the `networking/allow-ingress-access=true` label can receive network traffic.
Copy link

@asood-rh asood-rh Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
With the `deny-by-default` network policy in place, no pods can talk to each other or receive traffic from external sources. One option to enable communication is to allow some pods to receive traffic. To do so, you can create the following `ingress-access` network policy. With this network policy, pods with the `networking/allow-ingress-access=true` label can receive network traffic.
With the `deny-by-default` network policy, that denies both ingress and egress traffic, in place, no pods can talk to each other or receive traffic from external sources. One option to enable communication is to allow some pods to receive traffic. To do so, you can create the following `ingress-access` network policy. With this network policy, pods with the `networking/allow-ingress-access=true` label can receive network traffic.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevsmit If it is both ingress or egress pod traffic, it has to be default deny ingress and egress.
If it is about ingress then it should just refer to incoming traffic.


:_mod-docs-content-type: PROCEDURE
[id="nw-networkpolicy-configuring-internet-egress-pods_{context}"]
= Configuring internet egress for pods

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
= Configuring internet egress for pods
= Configuring external egress for pods


[WARNING]
====
Without configuring a `NetworkPolicy` custom resource (CR) that allows traffic communication, the following policy might cause communication problems across your cluster.
Copy link

@asood-rh asood-rh Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Without configuring a `NetworkPolicy` custom resource (CR) that allows traffic communication, the following policy might cause communication problems across your cluster.
Without configuring a `NetworkPolicy` that allows traffic communication, the following policy might cause communication problems across your cluster.

(B)ANP would be custom resouce, network policy is API.

+
[source,terminal]
----
$ oc apply -f allow-ingress-tp-new.yaml -n project-a

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc apply -f allow-ingress-tp-new.yaml -n project-a
$ oc apply -f allow-ingress-to-new.yaml -n project-a

+
[source,terminal]
----
$ oc apply -f allow-ingress-tp-new.yaml -n project-c

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc apply -f allow-ingress-tp-new.yaml -n project-c
$ oc apply -f allow-ingress-to-new.yaml -n project-c

- podSelector:
matchLabels:
networking/allow-all-connections: "true"
----

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the intent is to allow the ingress from new project-c then the policy needs to be as below in project-a

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-from-new
spec:
  ingress:
  - from:
    - namespaceSelector: {}
      podSelector:
        matchLabels:
          networking/allow-all-connections: "true"
  podSelector: {}
  policyTypes:
  - Ingress

ingress:
- from:
- podSelector: {}
----

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above for cross project ingress traffic

[id="nw-networkpolicy-configuring-internet-egress-pods_{context}"]
= Configuring internet egress for pods

With the deny all egress network policy created in a namespace, pods within that namespace are made incapable of reaching _out_ to the internet. In most cases, at least some pods within a namespace need to reach external traffic.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
With the deny all egress network policy created in a namespace, pods within that namespace are made incapable of reaching _out_ to the internet. In most cases, at least some pods within a namespace need to reach external traffic.
With the deny all egress network policy created in a namespace, pods within that namespace are made incapable of reaching _out_ to the internet. In most cases, at least some pods within a namespace need to reach external destinations.

$ oc apply -f internet-egress.yaml -n project-a
----

. Apply the network policy to the `project-b` namespace by entering the following command:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not need to add policy in project-b when demonstrating egress to external destination. Makes sense if egress to another namespace needs to be demonstrated

+
[source,terminal]
----
$ oc exec -it <pod_name> -n project-a -- nslookup google.com

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc exec -it <pod_name> -n project-a -- nslookup google.com
$ oc exec -it busybox-pod-a -n project-a -- nslookup google.com

As the pod was labelled in previous step.
$ oc exec -it test-pod-a -n project-a -- nslookup google.com can be used to demonstrate egress traffic does not work if the pod does not have a required label.


[WARNING]
====
Without configuring a `NetworkPolicy` custom resource (CR) that allows traffic communication, the following policy might cause communication problems across your cluster.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Without configuring a `NetworkPolicy` custom resource (CR) that allows traffic communication, the following policy might cause communication problems across your cluster.
Without configuring a `NetworkPolicy` that allows traffic communication, the following policy might cause communication problems across your cluster.


.Prerequisites

* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.
Copy link

@asood-rh asood-rh Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.
* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin.

The plugin is deprecated so should not mention it in latest documentation. The OVN-Kubernetes plugin does not have the mentioned mode.

;; connection timed out; no servers could be reached
----

. Test ingress connection between pods in the `project-a` and `project-b` namespaces by entering the following command. Because the `default-deny-all-egress` network policy breaks pod-to-pod communication for ingress, pods should not longer be able to communicate.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
. Test ingress connection between pods in the `project-a` and `project-b` namespaces by entering the following command. Because the `default-deny-all-egress` network policy breaks pod-to-pod communication for ingress, pods should not longer be able to communicate.
. Test ingress connection between pods in the `project-a` and `project-b` namespaces by entering the following command. Because the `default-deny-all-egress` network policy breaks pod-to-pod communication for egress, pods should not longer be able to communicate.


.Prerequisites

* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin, with `mode: NetworkPolicy` set.
* Your cluster uses a network plugin that supports `NetworkPolicy` objects, such as the OVN-Kubernetes network plugin.

[id="nw-networkpolicy-same-namespace-communication_{context}"]
= Creating a network policy for same namespace communication

With the `deny-by-default` network policy in place, a second option to enable communication is to create a network policy that allows for same namespace communication using the following `allow-same-namespace` network policy.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
With the `deny-by-default` network policy in place, a second option to enable communication is to create a network policy that allows for same namespace communication using the following `allow-same-namespace` network policy.
With the `deny-by-default` network policy in place, a second option to enable pod to pod communication within namespace is to create a network policy that allows for same namespace communication using the following `allow-same-namespace` network policy.

+
[source,terminal]
----
$ oc label namespace project-a networking/namespace=n1 --overwrite

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc label namespace project-a networking/namespace=n1 --overwrite
$ oc label namespace project-a networking/namespace=n1

--overwrite is not needed as the label does not exist on the namespace

+
[source,terminal]
----
$ oc label namespace project-b networking/namespace=n2 --overwrite

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc label namespace project-b networking/namespace=n2 --overwrite
$ oc label namespace project-b networking/namespace=n2

+
[source,terminal]
----
$ oc label pod busybox-pod-a app=send-data -n project-a

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc label pod busybox-pod-a app=send-data -n project-a
$ oc label pod busybox-pod-a app=send-data -n project-a --overwrite

[id="nw-networkpolicy-enable-pod-pod-communication-egress_{context}"]
= Enabling pod-to-pod communication

After creating a `default-deny-all` network policy that isolates pods for egress, pod-to-pod communication is broken for ingress. To re-establish ingress communication for pods, you must create a complementary egress network policy that _mirrors_ the network policy that allowed for ingress pod-to-pod communication across namespaces.
Copy link

@asood-rh asood-rh Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
After creating a `default-deny-all` network policy that isolates pods for egress, pod-to-pod communication is broken for ingress. To re-establish ingress communication for pods, you must create a complementary egress network policy that _mirrors_ the network policy that allowed for ingress pod-to-pod communication across namespaces.
After creating a egress `default-deny-all` network policy that isolates pods for egress, pod-to-pod communication is broken for ingress in the other namespace. To re-establish ingress communication for pods, you must create a complementary egress network policy that _mirrors_ the network policy that allowed for ingress pod-to-pod communication across namespaces.

- from:
- namespaceSelector:
matchLabels:
networking/namespace:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
networking/namespace:
networking/namespace: n1


.Prerequisites

* You have created the following `allow-n1-a-to-n2-b` ingress network policy:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* You have created the following `allow-n1-a-to-n2-b` ingress network policy:
* You have created the following `allow-n1-a-to-n2-b` ingress network policy in project-b:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-n1-a-to-n2-b-egress

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: allow-n1-a-to-n2-b-egress
name: egress-allow-n1-a-to-n2-b

+
[source,terminal]
----
$ oc apply -f allow-n1-a-to-n2-b-egress.yaml -n project-a

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc apply -f allow-n1-a-to-n2-b-egress.yaml -n project-a
$ oc apply -f egress-allow-n1-a-to-n2-b.yaml -n project-a

+
[source,terminal]
----
$ oc get pod -n project-a -o wide

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$ oc get pod -n project-a -o wide
$ oc get pod -n project-b -o wide

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch/enterprise-4.18 branch/enterprise-4.19 branch/enterprise-4.20 size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants