Skip to content

Default to SeaweedFS instead of MinIO #3240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 10, 2025
+23 −10
Merged

Default to SeaweedFS instead of MinIO #3240

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
68bdde1
use seaweedfs by default
akagami-harsh Sep 9, 2025
ec6ed71
fix unit test
akagami-harsh Sep 9, 2025
da6ca0a
rm redundant networkpolicy
akagami-harsh Sep 10, 2025
46b7a25
add back networkpolicy
akagami-harsh Sep 10, 2025
d14c7d1
add todo comment
akagami-harsh Sep 10, 2025
70bb210
sync seaweedfs manifests
akagami-harsh Sep 10, 2025
55f8942
fix typos
akagami-harsh Sep 10, 2025
9587f73
Update README.md
juliusvonkohout Sep 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,16 @@ kustomize build common/istio/kubeflow-istio-resources/base | kubectl apply -f -

Kubeflow Pipelines offers two deployment options to choose from, each designed for different use cases and operational preferences. The traditional database-based approach stores pipeline definitions in an external database, while the Kubernetes native API mode leverages Kubernetes custom resources for pipeline definition storage and management.

Note: Default artifact store (example install)
- The single-command installation using the `example` kustomization sets SeaweedFS as the default S3-compatible artifact store for Pipelines. It replaces `minio-service` to route S3 traffic to SeaweedFS and patches the Argo Workflow controller to use it.
- If you are following the step-by-step installation and want SeaweedFS as your Pipelines artifact store, apply the following overlay instead of the MinIO-based overlays:

```sh path=null start=null
kustomize build experimental/seaweedfs/istio | kubectl apply -f -
```

To switch back to MinIO, use the standard upstream Pipelines overlays shown below.


##### Pipeline Definitions Stored in the Database

Expand Down
6 changes: 4 additions & 2 deletions example/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@
- ../common/istio/istio-install/overlays/oauth2-proxy
# NOTE: For Google Kubernetes Engine (GKE), use:
# - ../common/istio/istio-install/overlays/gke
#GKE mounts `/opt/cni/bin` as read-only for security reasons, preventing the Istio CNI installer from writing the CNI binary.

Check warning on line 45 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

45:2 [comments] missing starting space in comment
#Use the GKE-specific overlay: `kubectl apply -k common/istio/istio-install/overlays/gke`.

Check warning on line 46 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

46:2 [comments] missing starting space in comment
#This overlay uses GKE's writable CNI directory at `/home/kubernetes/bin`.

Check warning on line 47 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

47:2 [comments] missing starting space in comment
#For more details, see [Istio CNI Prerequisites](https://istio.io/latest/docs/setup/additional-setup/cni/#prerequisites) and [Platform Prerequisites](https://istio.io/latest/docs/ambient/install/platform-prerequisites/)

Check warning on line 48 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

48:2 [comments] missing starting space in comment
# oauth2-proxy
# NOTE: only uncomment ONE of the following overlays, depending on your cluster type
- ../common/oauth2-proxy/overlays/m2m-dex-only # for all clusters
#- ../common/oauth2-proxy/overlays/m2m-dex-and-kind # for KIND clusters (allows K8S JWTs for gateway auth)

Check warning on line 52 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

52:2 [comments] missing starting space in comment
#- ../common/oauth2-proxy/overlays/m2m-dex-and-eks # for EKS clusters (NOTE: requires you to configure issuer, see overlay)

Check warning on line 53 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

53:2 [comments] missing starting space in comment
# Dex
- ../common/dex/overlays/oauth2-proxy
# KNative
Expand All @@ -66,8 +66,10 @@
- ../common/kubeflow-roles/base
# Kubeflow Istio Resources
- ../common/istio/kubeflow-istio-resources/base
# Kubeflow Pipelines
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please comment it out as in the other PR. and add a TODO to reenable this after the next kfp release as in the other PR.

- ../applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user # Pipeline Definitions Stored in the Database
# Kubeflow Pipelines (SeaweedFS default)
- ../experimental/seaweedfs/istio
# If you prefer MinIO instead, replace the line above with one of the following:
# - ../applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user # Pipeline Definitions Stored in the Database
# - ../applications/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user-k8s-native # Pipeline Definitions Stored as Kubernetes Resources
# Katib
- ../applications/katib/upstream/installs/katib-with-kubeflow
Expand All @@ -80,7 +82,7 @@
# Notebook Controller
- ../applications/jupyter/notebook-controller/upstream/overlays/kubeflow
# Profiles + KFAM with PSS (Pod Security Standards)
- ../applications/profiles/pss # TODO MUST BE UPSTREAMED

Check warning on line 85 in example/kustomization.yaml

View workflow job for this annotation

GitHub Actions / format_YAML_files 

85:32 [comments] too few spaces before comment: expected 2
# PVC Viewer
- ../applications/pvcviewer-controller/upstream/base
# Volumes Web App
Expand Down
2 changes: 1 addition & 1 deletion experimental/seaweedfs/base/seaweedfs/seaweedfs-networkpolicy.yaml
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can just delete the policy. We will anyway get it from common/networkpolicies by default.

Copy link
Contributor Author

@akagami-harsh akagami-harsh Sep 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test are failing cause when tests install pipelines form /experimental/seaweedfs/istio it dosent include networkpolicies from common folder. maybe we can keep the allow-same-namespace networkpolicy till the next kfp release, then kfp will have this networkpolicy by default

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-allow-same-namespace
name: default-allow-same-namespace-seaweedfs
namespace: kubeflow
spec:
podSelector: {}
Expand Down
Loading