|
4 | 4 | ## 1. Start a new minikube cluster
|
5 | 5 |
|
6 | 6 | [Minikube](https://minikube.sigs.k8s.io/docs/start/) can be installed via Homebrew or binary downloads.
|
| 7 | +Also make sure to enable the ingress and metrics plugin |
7 | 8 |
|
8 | 9 | ```shell
|
9 | 10 | minikube start -p trieve
|
| 11 | +minikube addons enable metrics-server -p trieve |
| 12 | +minikube addons enable ingress -p trieve |
10 | 13 | ```
|
11 | 14 |
|
12 | 15 | ## 2. Bootstrap glasskube
|
@@ -68,15 +71,64 @@ kubectl port-forward svc/trieve-keycloak-service 8080:8080
|
68 | 71 |
|
69 | 72 | Open [dashboard.localtrieve.com](http://dashboard.localtrieve.com) in your browser.
|
70 | 73 |
|
71 |
| -## Optional commands |
| 74 | +## Optional commands & debugging info |
72 | 75 |
|
73 |
| -#### Scale down all embedding servers: |
| 76 | +#### Working on low CPU machines |
| 77 | + |
| 78 | +You can scale down all embedding servers to reduce the memory and cpu usage of trieve. |
| 79 | + |
| 80 | +```shell |
| 81 | +kubectl scale deployment trieve-embedding-bgem3 trieve-embedding-jina trieve-embedding-reranker trieve-embedding-splade-doc trieve-embedding-splade-query --replicas 0 |
| 82 | +``` |
| 83 | + |
| 84 | +If you still want to do end-to-end test you can only start the `trieve-embedding-bgem3` |
| 85 | +and configure the dataset in a specific way: |
| 86 | + |
| 87 | +1. Create a new dataset |
| 88 | +2. Disable full text search |
| 89 | +3. Import test data |
| 90 | + |
| 91 | +After that the ingestion worker will orchestrate the ingestion. |
| 92 | + |
| 93 | +You can see the ingestion progress by connecting to the redis pod and execute the `redis-cli` and |
| 94 | +monitor progress by looking at the queues `llen ingestion` and `llen processing`. |
| 95 | + |
| 96 | +After ingestion is done you can search the dataset by only setting the search type to semantic so |
| 97 | +only bgem3 is used. |
| 98 | + |
| 99 | +#### Qdrant |
| 100 | + |
| 101 | +Qdrant throws some weird exceptions during the initial database migrations, but keeps working normally after. |
| 102 | + |
| 103 | +You can monitor the status of the collections by looking at the Qdrant ui: |
| 104 | + |
| 105 | +```shell |
| 106 | +glasskube open qdrant |
| 107 | +``` |
| 108 | + |
| 109 | +You can fetch the API key with: |
| 110 | + |
| 111 | +``` |
| 112 | +kubectl get secret trieve-qdrant-qdrant-apikey -o jsonpath='{.data.api-key}' | base64 -d |
| 113 | +``` |
| 114 | + |
| 115 | +#### ClickHouse |
| 116 | + |
| 117 | +In order to test if ClickHouse works you can connect to clickhouse via: |
74 | 118 |
|
75 | 119 | ```shell
|
76 |
| -kubectl scale deployment trieve-embedding-bgem3 trieve-embedding-jina trieve-embedding-reranker trieve-embedding-splade-doc trieve-embedding-splade-query --replicas 0 |
| 120 | +kubectl exec -it chi-trieve-clickhouse-cluster1-0-0-0 -- clickhouse-client |
| 121 | +``` |
| 122 | + |
| 123 | +The password is currently `password` but will be changed in the future. |
77 | 124 |
|
| 125 | +In order to see if the custom embedding function works you can execute: |
| 126 | + |
| 127 | +``` |
| 128 | +select embed_p('test'); |
78 | 129 | ```
|
79 | 130 |
|
| 131 | + |
80 | 132 | #### Deleting the cluster
|
81 | 133 |
|
82 | 134 | ```shell
|
|
0 commit comments