Skip to content

Commit 976f549

Browse files
Improving documentation for data denormalization
1 parent e32be01 commit 976f549

File tree

1 file changed

+26
-1
lines changed

1 file changed

+26
-1
lines changed

content/integrate/redis-data-integration/data-pipelines/data-denormalization.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,32 @@ called `InvoiceLineItems`:
3535

3636
{{< image filename="/images/rdi/ingest/nest-flow.webp" width="500px" >}}
3737

38-
You configure normalization with a `nest` block in the child entities' RDI job, as shown in this example:
38+
39+
When configuring normalization first you need to configure the parent entity to use JSON as the target data type. Add `data_type: json` to the parent job. Example:
40+
41+
```yaml
42+
# jobs/invoice.yaml
43+
source:
44+
server_name: chinook
45+
schema: public
46+
table: Invoice
47+
48+
output:
49+
- uses: redis.write
50+
with:
51+
# Setting the data type to json ensures that the parent object will be created in a way that supports nesting.
52+
data_type: json
53+
# Important: do not set a custom key for the parent entity.
54+
# When nesting the child object under the parent, the parent key is automatically calculated based on
55+
# the parent table name and the parent key field and if a custom key is set, it will cause a mismatch
56+
# between the key used to write the parent and the key used to write the child.
57+
58+
```
59+
60+
After that parent model is configured, it is time to configure the child entities. To do that use the `nest` block, as shown in this example:
3961

4062
```yaml
63+
# jobs/invoice_line.yaml
4164
source:
4265
server_name: chinook
4366
schema: public
@@ -52,6 +75,7 @@ output:
5275
table: Invoice
5376
nesting_key: InvoiceLineId # cannot be composite
5477
parent_key: InvoiceId # cannot be composite
78+
child_key: InvoiceId # optional, if different from parent_key
5579
path: $.InvoiceLineItems # path must start from document root ($)
5680
structure: map # only map supported for now
5781
on_update: merge # only merge supported for now
@@ -111,3 +135,4 @@ There are several important things to note when you use nesting:
111135
See the
112136
[Debezium PostgreSQL Connector Documentation](https://debezium.io/documentation/reference/connectors/postgresql.html#postgresql-replica-identity)
113137
for more information about this.
138+
- Changing the foreign key value of a child object results in the child object being added to the new parent, but the old parent is not updated.

0 commit comments

Comments
 (0)