Skip to content

Commit afaf46d

Browse files
DOC-4067 added FAQ and note about harmless OOM message
1 parent 170ab34 commit afaf46d

File tree

2 files changed

+20
-2
lines changed

2 files changed

+20
-2
lines changed

content/integrate/redis-data-integration/ingest/data-pipelines/data-pipelines.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -495,3 +495,10 @@ completely. For example, you might want to apply a new transformation to all the
495495
data or refresh the dataset if RDI is disconnected from the
496496
source for a long time. In situations like these, you can *reset* the pipeline back
497497
to the snapshot phase. When this is complete, the pipeline continues with CDC as usual.
498+
499+
{{<note>}}During the snapshot phase and occasionally during CDC, Debezium
500+
will sometimes log a message saying that RDI is out of memory. This is just
501+
an information message, not an error condition, and is typically harmless. See the
502+
[Ingest FAQ]({{< relref "/integrate/redis-data-integration/ingest/faq#rdi-oom" >}})
503+
for more information.
504+
{{</note>}}

content/integrate/redis-data-integration/ingest/faq.md

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,10 +59,21 @@ replica of an Active-Active replication setup or an Auto tiering database.
5959

6060
If you don't configure RDI to capture a specific set of tables in the schema then it will
6161
detect any new tables when they are added. Similarly, RDI will capture new table columns
62-
and changes to column names unless you configure it for a specfic set of columns.
62+
and changes to column names unless you configure it for a specific set of columns.
6363
Bear in mind that the Redis keys in the target database will change to reflect the
6464
new or renamed tables and columns.
6565

66+
## Should I be concerned when the log says RDI is out of memory? {#rdi-oom}
67+
68+
Sometimes the Debezium log will contain a message saying that RDI is out of
69+
memory. This is actually an information message, not an error. It means that
70+
RDI has used all available memory for its data streams and must now concentrate
71+
on processing the streams to make space for new data from the collector.
72+
The message is typically logged during the
73+
snapshot phase of the
74+
[ingest pipeline lifecycle]({{< relref "/integrate/redis-data-integration/ingest/data-pipelines/data-pipelines#ingest-pipeline-lifecycle" >}})
75+
but you may also sometimes see it during the CDC phase.
76+
6677
## What happens when RDI can't write to the target Redis database?
6778

6879
RDI will keep attempting to write the changes to the target and will also attempt
@@ -79,7 +90,7 @@ more space available.
7990
## What does RDI do if the data is corrupted or invalid?
8091

8192
The collector reports the data to RDI in a structured JSON format. If
82-
the structure of the JSON data is invalid or if there is a fatal bug in the tranformation
93+
the structure of the JSON data is invalid or if there is a fatal bug in the transformation
8394
job then RDI can't transform the data. When this happens, RDI will store the original data
8495
in a "dead letter queue" along with a message to say why it was rejected. The dead letter
8596
queue is stored as a capped stream in the RDI staging database. You can see its contents

0 commit comments

Comments
 (0)