Switch to Istio CNI by default

[madmecodes]

Switch to Istio CNI by default

This PR changes the default Istio installation to use Istio CNI instead of standard Istio.
Key benefits include:

• Eliminates the need for privileged Istio init containers
• Improves compatibility with Pod Security Standards (PSS)
• Enables native sidecars support for better init container network access

Changes include:

• Updated example/kustomization.yaml to use Istio CNI paths
• Updated README.md to reflect new installation paths
• Added note about Ray operator being configured for Istio CNI compatibility

This change is part of the broader Rootless Kubeflow initiative #2528
and follows up on previous work #3061.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign juliusvonkohout for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[madmecodes]

@juliusvonkohout For GCP hostPath check,

manifests/common/istio-cni-1-24/istio-install/base/install.yaml

Line 3009 in 9825950

path: /opt/cni/bin

am i suppose to spin the cluster in GCP/GKE ad check the paths? will that cost me?

[juliusvonkohout]

@juliusvonkohout For GCP hostPath check,

manifests/common/istio-cni-1-24/istio-install/base/install.yaml

Line 3009 in 9825950

path: /opt/cni/bin

am i suppose to spin the cluster in GCP/GKE ad check the paths? will that cost me?

I know the paths and that it is correct for gcp. The question is how we can enable both cni directories at the same time. Since the default ones are correct for Azure, Kind and others.

[juliusvonkohout]

See #3061 (comment). We need to support both paths at the same time somehow.

[madmecodes]

Okay, looking into this

See #3061 (comment). We need to support both paths at the same time somehow.

[madmecodes]

Why This Multi-Path Approach woukd Work?

The patch adds both the standard path (/opt/cni/bin) and the GCP-specific path (/home/kubernetes/bin) as separate volume mounts in the Istio CNI DaemonSet.

1. The CNI installer in the container will attempt to install the CNI binary to both paths
2. Regardless of whether the node is running on standard Kubernetes or GCP/GKE, one of the paths will exist and be the correct one.
3. If a path doesn't exist, mounting a non-existent hostPath simply results in an empty directory, which causes no harm.

@juliusvonkohout what do you think, can this be a potential approach?

[juliusvonkohout]

Why This Multi-Path Approach woukd Work?

The patch adds both the standard path (/opt/cni/bin) and the GCP-specific path (/home/kubernetes/bin) as separate volume mounts in the Istio CNI DaemonSet.

1. The CNI installer in the container will attempt to install the CNI binary to both paths

2. Regardless of whether the node is running on standard Kubernetes or GCP/GKE, one of the paths will exist and be the correct one.

3. If a path doesn't exist, mounting a non-existent hostPath simply results in an empty directory, which causes no harm.

@juliusvonkohout what do you think, can this be a potential approach?

Yes, could work. Do you mind testing it on GCP? I think there is a small free 4GB node available by default if you have a gmail adress. You jus tneed to install Istio, so it should be enough. I can then later also test on some GCP clusters.

[juliusvonkohout]

Do you mind fixing python3: can't open file '/home/runner/work/manifests/manifests/tests/gh-actions/test_pipeline.py': [Errno 2] No such file or directory in https://github.com/kubeflow/manifests/actions/runs/15084697684/job/42405783624?pr=3135 in a separate PR ? I think the file has just been renamed since we have v1 and v2 kfp tests. CC @kunal-511 to help

[kunal-511]

@madmecodes The test_pipeline.py has been changed to test_pipeline_v2.py in #3129

[madmecodes]

@madmecodes The test_pipeline.py has been changed to test_pipeline_v2.py in #3129

The test_pipeline.py has been changed to test_pipeline_v2.py in #3129

this is updated #3136