Feature: Add Istio Ambient Mode Support via Overlay Method

[madmecodes]

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

This PR implements Istio Ambient Mode support for Kubeflow deployments using the overlay pattern within the existing istio-install structure.

Ambient mode provides a sidecar-free service mesh solution that reduces resource overhead while maintaining full L4/L7 traffic processing capabilities.

For Standard Deployments:
In example/kustomization.yaml, replace:

• ../common/istio/istio-install/overlays/oauth2-proxy
  With:
• ../common/istio/istio-install/overlays/ambient

For GKE Deployments:
In example/kustomization.yaml, use:

• ../common/istio/istio-install/overlays/ambient-gke

For OAuth2-Proxy + Ambient:
In example/kustomization.yaml, use:

• ../common/istio/istio-install/overlays/ambient-oauth2-proxy

📦 Dependencies

none

🐛 Related Issues

none

✅ Contributor Checklist

• I have tested these changes with kustomize. See Installation Prerequisites.
• All commits are signed-off to satisfy the DCO check.
• I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

---

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign juliusvonkohout for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[madmecodes]

/retest

[madmecodes]

@juliusvonkohout ambient overlay working fine, i tested in GKE, will test on kind too, and share the logs and confirming testing steps, maybe we can create pipeline test too.

[juliusvonkohout]

can we use kustomize components instead of overlays for GKE to reduce the complexity ?

[madmecodes]

can we use kustomize components instead of overlays for GKE to reduce the complexity ?

Yes, we can do that, also for this PR we can use inheritance the same way we do for base installation right now, and if we want component, we can create a followup PR for the same.
For this PR we can go something like this:

## overlays/ambient-gke/kustomization.yaml
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization

  resources:
  - ../ambient              ### <-Inherit from ambient overlay!

  patches:
  - path: gke-cni-patch.yaml
    target:
      kind: DaemonSet
      name: istio-cni-node
      namespace: kube-system
  - path: gke-ztunnel-patch.yaml
    target:
      kind: DaemonSet
      name: ztunnel
      namespace: istio-system

[juliusvonkohout]

can we use kustomize components instead of overlays for GKE to reduce the complexity ?

Yes, we can do that, also for this PR we can use inheritance the same way we do for base installation right now, and if we want component, we can create a followup PR for the same. For this PR we can go something like this:

## overlays/ambient-gke/kustomization.yaml
  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization

  resources:
  - ../ambient              ### <-Inherit from ambient overlay!

  patches:
  - path: gke-cni-patch.yaml
    target:
      kind: DaemonSet
      name: istio-cni-node
      namespace: kube-system
  - path: gke-ztunnel-patch.yaml
    target:
      kind: DaemonSet
      name: ztunnel
      namespace: istio-system

Yes we can make all GKE patches kustomize components in this PR here, also for istio-cni. But it should be a kustomize component without inheritance via - ../ambient ### <-Inherit from ambient overlay! We want to avoid inheritance for GKE stuff, that is the goal.

[juliusvonkohout]

See also https://www.fosstechnix.com/kubernetes-customization-with-kustomize-components/

[madmecodes]

/retest

[juliusvonkohout]

please also use a GKE kustomize component for istio-cni

[madmecodes]

/retest

[madmecodes]

I am also wondering where https://github.com/LogicalGuy77/manifests-logical/blob/acc205c3275dc40ec68bceb8bda33d32b95cb879/tests/kserve_jwt_authentication_test.sh#L15 is coming from. That line looks dangerous. Why did you add it in the first place?

I added it because the test needs a ServiceAccount to generate JWT tokens (kubectl create token default-editor), and without it the test was failing with "serviceaccount not found." (as i remember, in PR #3180 )

I used --dry-run=client | kubectl apply to make it idempotent, but I'm not sure if this is the right approach for tests. Should the test assume default-editor already exists (created by profile controller or elsewhere)?

Or is there a better pattern for creating test service accounts?

Happy to revise if there's a more appropriate way to handle this.

[madmecodes]

Also https://github.com/kubeflow/manifests/blob/master/.github/workflows/kserve_test.yaml is not properly merged. you just tripled the code. and it is not used in the full end-to-end test

Most of that belongs in the actual test.sh files. and it must be included here in a minimal and elegant fashion

manifests/.github/workflows/full_kubeflow_integration_test.yaml

Line 197 in 66aa578

- name: Run KServe Test

Created the new PR for the same, sorry for the overlook, #3254

[juliusvonkohout]

I am also wondering where https://github.com/LogicalGuy77/manifests-logical/blob/acc205c3275dc40ec68bceb8bda33d32b95cb879/tests/kserve_jwt_authentication_test.sh#L15 is coming from. That line looks dangerous. Why did you add it in the first place?

I added it because the test needs a ServiceAccount to generate JWT tokens (kubectl create token default-editor), and without it the test was failing with "serviceaccount not found." (as i remember, in PR #3180 )

I used --dry-run=client | kubectl apply to make it idempotent, but I'm not sure if this is the right approach for tests. Should the test assume default-editor already exists (created by profile controller or elsewhere)?

Or is there a better pattern for creating test service accounts?

Happy to revise if there's a more appropriate way to handle this.

The service account is created automatically by installing the multi-tenancy component and then creating the Kubeflow profile. SO please do not create it manually. Just by creating the profile.

[juliusvonkohout]

Can we deploy Ztunnel in another namespace called istio-ztunnel or so? So separate and isolated ?

[juliusvonkohout]

See also #3246 (comment)

[juliusvonkohout]

will this be covered by the istio upgrade script https://github.com/kubeflow/manifests/blob/master/scripts/synchronize-istio-manifests.sh ? CC @kunal-511

[kunal-511]

we have to the update the https://github.com/kubeflow/manifests/blob/master/scripts/synchronize-istio-manifests.sh to handle and include ztunnel manifest generation

[juliusvonkohout]

I see

1. https://github.com/kubeflow/manifests/pull/3246/files/f4776546cf1755d33ddd9bb17cedd3980a7e6881#r2410641900 so maybe istio-ztunnel if kube-system does not work.

2. https://github.com/kubeflow/manifests/pull/3246/files#r2410712042 istio upgrade script compatibility