You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The `processors` section configures the behavior of the pipeline. The [example](#example)
243
+
configuration above contains the following properties:
244
+
245
+
-`on_failed_retry_interval`: Number of seconds to wait before retrying a failed operation.
246
+
The default is 5 seconds.
247
+
-`read_batch_size`: Maximum number of records to read from the source database. RDI will
248
+
wait for the batch to fill up to `read_batch_size` or for `duration` to elapse,
249
+
whichever happens first. The default is 2000.
250
+
-`duration`: Time (in ms) after which data will be read from the stream even if
251
+
`read_batch_size` was not reached. The default is 100 ms.
252
+
-`write_batch_size`: The batch size for writing data to the target Redis database. This should be
253
+
less than or equal to the `read_batch_size`. The default is 200.
254
+
-`dedup`: Boolean value to enable the deduplication mechanism. The default is `false`.
255
+
-`dedup_max_size`: Maximum size of the deduplication set. The default is 1024.
256
+
-`error_handling`: The strategy to use when an invalid record is encountered. The available
257
+
strategies are `ignore` and `dlq` (store rejected messages in a dead letter queue).
258
+
The default is `dlq`. See
259
+
[What does RDI do if the data is corrupted or invalid?]({{< relref "/integrate/redis-data-integration/faq#what-does-rdi-do-if-the-data-is-corrupted-or-invalid" >}})
0 commit comments