You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
`employeeid` field to `id` and create two nested objects for the `address`
48
-
and `contact` information. The `name` field is the concatenation of the existing
49
-
`firstname` and `lastname` fields, with `lastname` converted to uppercase.
50
-
In the `contact` subobject, the `email` address is obfuscated slightly, using the
51
-
`replace()` function to hide the '@' sign and dots.
52
-
53
-
In the `output` section of the job file, we specify that we want to write
54
-
to a JSON object with a custom key. Note that in the `output` section, you must refer to
55
-
fields defined in the `map` transformation, so we use the new name `id`
56
-
for the key instead of `employeeid`.
27
+
transformation, as described in the following sections.
57
28
58
-
The full example is shown below:
29
+
## Creating multilevel JSON objects
30
+
31
+
You can use the `map` transformation to create a new structure for the output data, which can include nested objects and calculated fields. The `map` transformation allows you to define a new structure using an expression language, such as SQL or JavaScript.
59
32
60
33
```yaml
61
34
source:
62
35
db: chinook
63
36
table: employee
37
+
64
38
transform:
65
39
- uses: map
66
40
with:
@@ -81,16 +55,30 @@ transform:
81
55
}
82
56
}
83
57
language: jmespath
58
+
84
59
output:
85
60
- uses: redis.write
86
61
with:
87
-
connection: target
88
62
data_type: json
89
63
key:
90
64
expression: concat(['emp:', id])
91
65
language: jmespath
92
66
```
93
67
68
+
69
+
The example above creates a new JSON object with the following structure:
70
+
- A top-level `id` field that is the same as the `employeeid` field in the source table.
71
+
- A `name` field that is a concatenation of the `firstname` and `lastname` fields, with the `lastname` converted to uppercase.
72
+
- An `address` subobject that contains the `address`, `city`, `state`, `postalcode`, and `country` fields.
73
+
- A `contact` subobject that contains the `phone` field and a modified version of the `email` field, where the '@' sign and dots are replaced with '_at_' and '_dot_' respectively.
74
+
75
+
The `output` section of the file configures the job to write
76
+
to a JSON object with a custom key. Note that in the `output` section, you must refer to
77
+
fields defined in the `map` transformation, so we use the new name `id`
78
+
for the key instead of `employeeid`.
79
+
80
+
81
+
94
82
If you query one of the new JSON objects, you see output like the following:
95
83
96
84
```bash
@@ -118,7 +106,7 @@ Formatted in the usual JSON style, the output looks like the sample below:
118
106
}
119
107
```
120
108
121
-
## Map to a hash structure
109
+
## Creating hash structure
122
110
123
111
This example creates a new [hash]({{< relref "/develop/data-types/hashes" >}})
124
112
object structure for items from the `track` table. Here, the `map` transformation uses
summary: Redis Data Integration keeps Redis in sync with the primary database in near
14
+
real time.
15
+
type: integration
16
+
weight: 40
17
+
---
18
+
19
+
Sometimes, you may want to remap the fields in the output of a data pipeline. You can do this by defining a `mapping` section in the output configuration.
20
+
21
+
```yaml
22
+
source:
23
+
table: Customer
24
+
25
+
output:
26
+
- uses: redis.write
27
+
with:
28
+
data_type: hash
29
+
mapping:
30
+
- CustomerId: id
31
+
- FirstName: first_name
32
+
- LastName: last_name
33
+
```
34
+
35
+
The example above remaps the `CustomerId` field to `id`, `FirstName` to `first_name`, and `LastName` to `last_name` in the output. This allows you to customize the field names in the Redis data store according to your application's requirements.
36
+
You can also use `mapping` to include only the fields you need in the output and exclude the rest.
37
+
38
+
Mapping only allows you to rename fields and limit the output to specific fields and define a single level structure. To create nested structures and/or perform operations on the field values you can use the [map transformation]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/map-example" >}}).
0 commit comments