Skip to content

Commit c763690

Browse files
vhristev-1vhristev-1
committed
RDSC-3487: Update RDI -> Deploy a pipeline documentation
1 parent b4ab052 commit c763690

File tree

1 file changed

+66
-84
lines changed
  • content/integrate/redis-data-integration/data-pipelines

1 file changed

+66
-84
lines changed

‎content/integrate/redis-data-integration/data-pipelines/deploy.md

Lines changed: 66 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -62,116 +62,98 @@ redis-di set-secret SOURCE_DB_USERNAME myUserName
6262
```
6363

6464
### Set secrets for K8s/Helm deployment
65-
66-
Use
67-
[`kubectl create secret generic`](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_secret_generic/)
68-
to set secrets for a K8s/Helm deployment. The general pattern of the commands is:
69-
65+
66+
Use the provided `scripts/rdi-secret.sh` shell script to set the specified secrets. The general pattern to use it is:
7067
```bash
71-
kubectl create secret generic <DB> \
72-
--namespace=rdi \
73-
--from-literal=<SECRET-NAME>=<SECRET-VALUE>
68+
scripts/rdi-secret.sh set <SECRET-KEY> <SECRET-VALUE>
7469
```
7570

76-
Where `<DB>` is either `source-db` for source secrets or `target-db` for target secrets.
77-
78-
If you use TLS or mTLS for either the source or target databases, you also need to create the `source-db-ssl` and/or `target-db-ssl` K8s secrets that contain the certificates used to establish secure connections. The general pattern of the commands is:
79-
71+
The script offers functionality to retrieve a specific secret, as well as the capability to list all available secrets within the system:
8072
```bash
81-
kubectl create secret generic <DB>-ssl \
82-
--namespace=rdi \
83-
--from-file=<FILE-NAME>=<FILE-PATH>
73+
# Get specific secret
74+
scripts/rdi-secret.sh set <SECRET-KEY>
75+
76+
# List all secrets
77+
scripts/rdi-secret.sh list
8478
```
8579

86-
When you create these secrets, ensure that all certificates and keys are in `PEM` format. The only exception to this is that for PostgreSQL, the private key in the `source-db-ssl` secret (the `client.key` file) must be in `DER` format. If you have a key in `PEM` format, you must convert it to `DER` before creating the `source-db-ssl` secret using the command:
80+
When you create secrets for TLS or mTLS, ensure that all certificates and keys are in `PEM` format. The only exception to this is that for PostgreSQL, the private key `SOURCE_DB_KEY` secret (the `client.key` file) must be in `DER` format. If you have a key in `PEM` format, you must convert it to `DER` before creating the `SOURCE_DB_KEY` secret using the command:
8781

8882
```bash
8983
openssl pkcs8 -topk8 -inform PEM -outform DER -in /path/to/myclient.key -out /path/to/myclient.pk8 -nocrypt
9084
```
9185

9286
This command assumes that the private key is not encrypted. See the [`openssl` documentation](https://docs.openssl.org/master/) to learn how to convert an encrypted private key.
93-
87+
9488
The specific command lines for source secrets are as follows:
9589

9690
```bash
9791
# Without source TLS
98-
# Create or update source-db secret
99-
kubectl create secret generic source-db --namespace=rdi \
100-
--from-literal=SOURCE_DB_USERNAME=yourUsername \
101-
--from-literal=SOURCE_DB_PASSWORD=yourPassword \
102-
--save-config --dry-run=client -o yaml | kubectl apply -f -
92+
scripts/rdi-secret.sh set SOURCE_DB_USERNAME yourUsername
93+
scripts/rdi-secret.sh set SOURCE_DB_PASSWORD yourPassword
94+
# Verify that the secrets are created/updated
95+
scripts/rdi-secret.sh get SOURCE_DB_USERNAME
96+
scripts/rdi-secret.sh get SOURCE_DB_PASSWORD
10397

10498
# With source TLS
105-
# Create of update source-db secret
106-
kubectl create secret generic source-db --namespace=rdi \
107-
--from-literal=SOURCE_DB_USERNAME=yourUsername \
108-
--from-literal=SOURCE_DB_PASSWORD=yourPassword \
109-
--from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \
110-
--save-config --dry-run=client -o yaml | kubectl apply -f -
111-
# Create or update source-db-ssl secret
112-
kubectl create secret generic source-db-ssl --namespace=rdi \
113-
--from-file=ca.crt=/path/to/myca.crt \
114-
--save-config --dry-run=client -o yaml | kubectl apply -f -
99+
scripts/rdi-secret.sh set SOURCE_DB_USERNAME yourUsername
100+
scripts/rdi-secret.sh set SOURCE_DB_PASSWORD yourPassword
101+
scripts/rdi-secret.sh set SOURCE_DB_CACERT /path/to/myca.crt
102+
# Verify that the secrets are created/updated
103+
scripts/rdi-secret.sh get SOURCE_DB_USERNAME
104+
scripts/rdi-secret.sh get SOURCE_DB_PASSWORD
105+
scripts/rdi-secret.sh get SOURCE_DB_CACERT
115106

116107
# With source mTLS
117-
# Create or update source-db secret
118-
kubectl create secret generic source-db --namespace=rdi \
119-
--from-literal=SOURCE_DB_USERNAME=yourUsername \
120-
--from-literal=SOURCE_DB_PASSWORD=yourPassword \
121-
--from-literal=SOURCE_DB_CACERT=/etc/certificates/source_db/ca.crt \
122-
--from-literal=SOURCE_DB_CERT=/etc/certificates/source_db/client.crt \
123-
--from-literal=SOURCE_DB_KEY=/etc/certificates/source_db/client.key \
124-
--from-literal=SOURCE_DB_KEY_PASSWORD=yourKeyPassword \ # add this only if SOURCE_DB_KEY is password-protected
125-
--save-config --dry-run=client -o yaml | kubectl apply -f -
126-
# Create or update source-db-ssl secret
127-
kubectl create secret generic source-db-ssl --namespace=rdi \
128-
--from-file=ca.crt=/path/to/myca.crt \
129-
--from-file=client.crt=/path/to/myclient.crt \
130-
--from-file=client.key=/path/to/myclient.key \
131-
--save-config --dry-run=client -o yaml | kubectl apply -f -
108+
scripts/rdi-secret.sh set SOURCE_DB_USERNAME yourUsername
109+
scripts/rdi-secret.sh set SOURCE_DB_PASSWORD yourPassword
110+
scripts/rdi-secret.sh set SOURCE_DB_CACERT /path/to/myca.crt
111+
scripts/rdi-secret.sh set SOURCE_DB_CERT /path/to/myclient.crt
112+
scripts/rdi-secret.sh set SOURCE_DB_KEY /path/to/myclient.key
113+
scripts/rdi-secret.sh set SOURCE_DB_KEY_PASSWORD yourKeyPassword # add this only if SOURCE_DB_KEY is password-protected
114+
# Verify that the secrets are created/updated
115+
scripts/rdi-secret.sh get SOURCE_DB_USERNAME
116+
scripts/rdi-secret.sh get SOURCE_DB_PASSWORD
117+
scripts/rdi-secret.sh get SOURCE_DB_CACERT
118+
scripts/rdi-secret.sh get SOURCE_DB_CERT
119+
scripts/rdi-secret.sh get SOURCE_DB_KEY
120+
scripts/rdi-secret.sh get SOURCE_DB_KEY_PASSWORD
132121
```
133122

134123
The corresponding command lines for target secrets are:
135124

136125
```bash
137-
# Without target TLS
138-
# Create or update target-db secret
139-
kubectl create secret generic target-db --namespace=rdi \
140-
--from-literal=TARGET_DB_USERNAME=yourUsername \
141-
--from-literal=TARGET_DB_PASSWORD=yourPassword \
142-
--save-config --dry-run=client -o yaml | kubectl apply -f -
143-
144-
# With target TLS
145-
# Create of update target-db secret
146-
kubectl create secret generic target-db --namespace=rdi \
147-
--from-literal=TARGET_DB_USERNAME=yourUsername \
148-
--from-literal=TARGET_DB_PASSWORD=yourPassword \
149-
--from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \
150-
--save-config --dry-run=client -o yaml | kubectl apply -f -
151-
# Create or update target-db-ssl secret
152-
kubectl create secret generic target-db-ssl --namespace=rdi \
153-
--from-file=ca.crt=/path/to/myca.crt \
154-
--save-config --dry-run=client -o yaml | kubectl apply -f -
155-
156-
# With target mTLS
157-
# Create or update target-db secret
158-
kubectl create secret generic target-db --namespace=rdi \
159-
--from-literal=TARGET_DB_USERNAME=yourUsername \
160-
--from-literal=TARGET_DB_PASSWORD=yourPassword \
161-
--from-literal=TARGET_DB_CACERT=/etc/certificates/target_db/ca.crt \
162-
--from-literal=TARGET_DB_CERT=/etc/certificates/target_db/client.crt \
163-
--from-literal=TARGET_DB_KEY=/etc/certificates/target_db/client.key \
164-
--from-literal=TARGET_DB_KEY_PASSWORD=yourKeyPassword \ # add this only if TARGET_DB_KEY is password-protected
165-
--save-config --dry-run=client -o yaml | kubectl apply -f -
166-
# Create or update target-db-ssl secret
167-
kubectl create secret generic target-db-ssl --namespace=rdi \
168-
--from-file=ca.crt=/path/to/myca.crt \
169-
--from-file=client.crt=/path/to/myclient.crt \
170-
--from-file=client.key=/path/to/myclient.key \
171-
--save-config --dry-run=client -o yaml | kubectl apply -f -
172-
```
126+
# Without source TLS
127+
scripts/rdi-secret.sh set TARGET_DB_USERNAME yourUsername
128+
scripts/rdi-secret.sh set TARGET_DB_PASSWORD yourPassword
129+
# Verify that the secrets are created/updated
130+
scripts/rdi-secret.sh get TARGET_DB_USERNAME
131+
scripts/rdi-secret.sh get TARGET_DB_PASSWORD
173132

174-
Note that the certificate paths contained in the secrets `SOURCE_DB_CACERT`, `SOURCE_DB_CERT`, and `SOURCE_DB_KEY` (for the source database) and `TARGET_DB_CACERT`, `TARGET_DB_CERT`, and `TARGET_DB_KEY` (for the target database) are internal to RDI, so you *must* use the values shown in the example above. You should only change the certificate paths when you create the `source-db-ssl` and `target-db-ssl` secrets.
133+
# With source TLS
134+
scripts/rdi-secret.sh set TARGET_DB_USERNAME yourUsername
135+
scripts/rdi-secret.sh set TARGET_DB_PASSWORD yourPassword
136+
scripts/rdi-secret.sh set TARGET_DB_CACERT /path/to/myca.crt
137+
# Verify that the secrets are created/updated
138+
scripts/rdi-secret.sh get TARGET_DB_USERNAME
139+
scripts/rdi-secret.sh get TARGET_DB_PASSWORD
140+
scripts/rdi-secret.sh get TARGET_DB_CACERT
141+
142+
# With source mTLS
143+
scripts/rdi-secret.sh set TARGET_DB_USERNAME yourUsername
144+
scripts/rdi-secret.sh set TARGET_DB_PASSWORD yourPassword
145+
scripts/rdi-secret.sh set TARGET_DB_CACERT /path/to/myca.crt
146+
scripts/rdi-secret.sh set TARGET_DB_CERT /path/to/myclient.crt
147+
scripts/rdi-secret.sh set TARGET_DB_KEY /path/to/myclient.key
148+
scripts/rdi-secret.sh set TARGET_DB_KEY_PASSWORD yourKeyPassword # add this only if TARGET_DB_KEY is password-protected
149+
# Verify that the secrets are created/updated
150+
scripts/rdi-secret.sh get TARGET_DB_USERNAME
151+
scripts/rdi-secret.sh get TARGET_DB_PASSWORD
152+
scripts/rdi-secret.sh get TARGET_DB_CACERT
153+
scripts/rdi-secret.sh get TARGET_DB_CERT
154+
scripts/rdi-secret.sh get TARGET_DB_KEY
155+
scripts/rdi-secret.sh get TARGET_DB_KEY_PASSWORD
156+
```
175157

176158
## Deploy a pipeline
177159

0 commit comments

Comments
 (0)