You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After you deploy a pipeline, it goes through the following phases:
72
+
73
+
1.*Deploy* - when you deploy the pipeline, RDI first validates it before use.
74
+
Then, the [operator]({{< relref "/integrate/redis-data-integration/architecture#how-rdi-is-deployed">}}) creates and configures the collector and stream processor that will run the pipeline.
75
+
1.*Snapshot* - The collector starts the pipeline by creating a snapshot of the full
76
+
dataset. This involves reading all the relevant source data, transforming it and then
77
+
writing it into the Redis target. This phase typically takes minutes to
78
+
hours if you have a lot of data.
79
+
1.*CDC* - Once the snapshot is complete, the collector starts listening for updates to
80
+
the source data. Whenever a change is committed to the source, the collector captures
81
+
it and adds it to the target through the pipeline. This phase continues indefinitely
82
+
unless you change the pipeline configuration.
83
+
1.*Update* - If you update the pipeline configuration, the operator applies it
84
+
to the collector and the stream processor. Note that the changes only affect newly-captured
85
+
data unless you reset the pipeline completely. Once RDI has accepted the updates, the
86
+
pipeline returns to the CDC phase with the new configuration.
87
+
1.*Reset* - There are circumstances where you might want to rebuild the dataset
88
+
completely. For example, you might want to apply a new transformation to all the source
89
+
data or refresh the dataset if RDI is disconnected from the
90
+
source for a long time. In situations like these, you can *reset* the pipeline back
91
+
to the snapshot phase. When this is complete, the pipeline continues with CDC as usual.
92
+
93
+
## Using a pipeline
94
+
95
+
Follow the steps described in the sections below to prepare and run an RDI pipeline.
96
+
97
+
### 1. Prepare the source database
98
+
99
+
Before using the pipeline you must first prepare your source database to use
100
+
the Debezium connector for *change data capture (CDC)*. See the
0 commit comments