Skip to content

Commit 12394f6

Browse files
kunal-511juliusvonkohout
kunal-511
and
juliusvonkohout
authored
Add KFP V1 Pipeline Test to CI Workflow (#3127)
* Added KFP V1 Pipeline compatibility test Signed-off-by: kunal-511 <yoyokvunal@gmail.com> * Added argument parameter Signed-off-by: kunal-511 <yoyokvunal@gmail.com> * fixed to use correct kfpV1 syntax Signed-off-by: kunal-511 <yoyokvunal@gmail.com> * Update pipeline_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update full_kubeflow_integration_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update pipeline_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update requirements.txt Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update full_kubeflow_integration_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update full_kubeflow_integration_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update pipeline_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update pipeline_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update pipeline_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update pipeline_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> * Update full_kubeflow_integration_test.yaml Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> --------- Signed-off-by: kunal-511 <yoyokvunal@gmail.com> Signed-off-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> Co-authored-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com>
1 parent cb79468 commit 12394f6

File tree

4 files changed

+73
-16
lines changed

4 files changed

+73
-16
lines changed

‎.github/workflows/full_kubeflow_integration_test.yaml

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -100,9 +100,6 @@ jobs:
100100
- name: Wait for All Pods to be Ready
101101
run: kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout 60s --field-selector=status.phase!=Succeeded
102102

103-
- name: Install Dependencies
104-
run: pip install pytest kubernetes kfp==2.13.0 kserve pytest-timeout pyyaml requests
105-
106103
- name: Port-forward the istio-ingress gateway
107104
run: ./tests/gh-actions/port_forward_gateway.sh
108105

@@ -112,13 +109,18 @@ jobs:
112109
- name: Test Dex Login
113110
run: |
114111
pip3 install -q requests
115-
116112
python3 tests/gh-actions/test_dex_login.py
117-
118113
echo "Dex login test completed successfully."
119114
120-
- name: Test Pipeline Access with Authorized Token
115+
- name: V1 Pipeline Test
116+
run: |
117+
pip3 install "kfp>=1.8.22,<2.0.0"
118+
TOKEN="$(kubectl -n $KF_PROFILE create token default-editor)"
119+
python3 tests/gh-actions/test_pipeline_v1.py "${TOKEN}" "${KF_PROFILE}"
120+
121+
- name: V2 Pipeline Test
121122
run: |
123+
pip3 install -U "kfp>=2.13.0"
122124
TOKEN="$(kubectl -n $KF_PROFILE create token default-editor)"
123125
python3 tests/gh-actions/test_pipeline.py run_pipeline "${TOKEN}" "${KF_PROFILE}"
124126

‎.github/workflows/pipeline_test.yaml

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,21 @@ on:
33
pull_request:
44
paths:
55
- tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh
6-
- .github/workflows/test_pipeline.yaml
6+
- .github/workflows/pipeline_test.yaml
77
- apps/pipeline/upstream/**
8-
- tests/gh-actions/install_istio.sh
8+
- tests/gh-actions/install_istio*.sh
99
- tests/gh-actions/install_cert_manager.sh
1010
- tests/gh-actions/install_oauth2-proxy.sh
1111
- common/cert-manager/**
1212
- common/oauth2-proxy/**
1313
- common/istio*/**
1414
- tests/gh-actions/test_pipeline.py
15+
- tests/gh-actions/test_pipeline_v1.py
1516
- experimental/security/PSS/*
1617

18+
env:
19+
KF_PROFILE: kubeflow-user-example-com
20+
1721
jobs:
1822
build:
1923
timeout-minutes: 15
@@ -54,7 +58,6 @@ jobs:
5458

5559
- name: Verify Pipeline Integration
5660
run: |
57-
KF_PROFILE=kubeflow-user-example-com
5861
if ! kubectl get secret mlpipeline-minio-artifact -n $KF_PROFILE > /dev/null 2>&1; then
5962
echo "Error: Secret mlpipeline-minio-artifact not found in namespace $KF_PROFILE"
6063
exit 1
@@ -64,17 +67,21 @@ jobs:
6467
- name: Port forward
6568
run: ./tests/gh-actions/port_forward_gateway.sh
6669

67-
- name: List and deploy test pipeline with authorized ServiceAccount Token
70+
- name: V1 Pipeline Test
6871
run: |
69-
pip3 install kfp==2.13.0
70-
KF_PROFILE=kubeflow-user-example-com
72+
pip3 install "kfp>=1.8.22,<2.0.0"
73+
TOKEN="$(kubectl -n $KF_PROFILE create token default-editor)"
74+
python3 tests/gh-actions/test_pipeline_v1.py "${TOKEN}" "${KF_PROFILE}"
75+
76+
- name: V2 Pipeline Test
77+
run: |
78+
pip3 install -U kfp==2.13.0
7179
TOKEN="$(kubectl -n $KF_PROFILE create token default-editor)"
7280
python3 tests/gh-actions/test_pipeline.py run_pipeline "${TOKEN}" "${KF_PROFILE}"
7381
7482
- name: Fail to list pipelines with unauthorized ServiceAccount Token
7583
run: |
76-
pip3 install kfp==2.13.0
77-
KF_PROFILE=kubeflow-user-example-com
84+
pip3 install -U kfp==2.13.0
7885
TOKEN="$(kubectl -n default create token default)"
7986
python3 tests/gh-actions/test_pipeline.py test_unauthorized_access "${TOKEN}" "${KF_PROFILE}"
8087
echo "Test succeeded. Token from unauthorized ServiceAccount cannot list pipelines in $KF_PROFILE namespace."
@@ -94,4 +101,3 @@ jobs:
94101
95102
- name: Applying Pod Security Standards restricted levels for static namespaces
96103
run: ./tests/gh-actions/enable_restricted_PSS.sh
97-

‎tests/gh-actions/kserve/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
pytest>=7.0.0
2-
kserve>=0.14.1
2+
kserve>=0.15.0
33
kubernetes>=18.20.0
44
requests>=2.18.4

‎tests/gh-actions/test_pipeline_v1.py

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
#!/usr/bin/env python3
2+
3+
import kfp
4+
import sys
5+
import time
6+
7+
def hello_world_op():
8+
from kfp.components import func_to_container_op
9+
10+
def hello_world():
11+
print("Hello World from Kubeflow Pipelines V1!")
12+
return "Hello World"
13+
14+
return func_to_container_op(hello_world)
15+
16+
def hello_world_pipeline():
17+
hello_op = hello_world_op()
18+
hello_op()
19+
20+
def run_v1_pipeline(token, namespace):
21+
client = kfp.Client(host="http://localhost:8080/pipeline", existing_token=token)
22+
23+
experiment = client.create_experiment("v1-pipeline-test", namespace=namespace)
24+
25+
pipeline_run = client.create_run_from_pipeline_func(
26+
hello_world_pipeline,
27+
experiment_name=experiment.name,
28+
run_name="v1-hello-world",
29+
namespace=namespace,
30+
arguments={}
31+
)
32+
33+
for iteration in range(15):
34+
pipeline_status = client.get_run(pipeline_run.run_id).run.status
35+
36+
if pipeline_status == "Succeeded":
37+
return
38+
elif pipeline_status not in ["Running", "Pending"]:
39+
sys.exit(1)
40+
41+
time.sleep(10)
42+
43+
sys.exit(1)
44+
45+
if __name__ == "__main__":
46+
if len(sys.argv) != 3:
47+
sys.exit(1)
48+
49+
run_v1_pipeline(sys.argv[1], sys.argv[2])

0 commit comments

Comments
 (0)