@@ -31,15 +31,16 @@ database (the optional `db` value here corresponds to the
31
31
` sources.<source-name>.connection.database ` value defined in
32
32
[ ` config.yaml ` ] ({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines#the-configyaml-file" >}})).
33
33
34
- In the ` transform ` section, the ` add_field ` transformation adds an extra field called ` fullname `
35
- to the object, which is created by concatenating the existing ` firstname ` and ` lastname ` fields using
36
- a [ SQL expression ] ( https://www.simplilearn.com/tutorials/sql-tutorial/concat-function-in-sql ) .
37
- You can also specify ` jmespath ` as the ` language ` if you prefer to create the new
38
- field with a [ JMESPath ] ( https://jmespath. org/ ) expression
34
+ In the ` transform ` section, the ` add_field ` transformation adds an extra field called ` localphone `
35
+ to the object, which is created by removing the country and area code from the ` phone `
36
+ field with the ` regex_replace() ` [ JMESPath ] ( https://jmespath.org/ ) function .
37
+ You can also specify ` sql ` as the ` language ` if you prefer to create the new
38
+ field with an [ SQL ] ( https://en.wikipedia. org/wiki/SQL ) expression.
39
39
40
40
The ` output ` section specifies ` hash ` as the ` data_type ` to write to the target, which
41
41
overrides the default setting of ` target_data_type ` defined in ` config.yaml ` . Also, the
42
- ` output.with.key ` section specifies a custom key format of the form ` cust:<customerid> ` .
42
+ ` output.with.key ` section specifies a custom key format of the form ` cust:<id> ` where
43
+ the ` id ` part is generated by the ` uuid() ` function.
43
44
44
45
The full example is shown below:
45
46
@@ -50,16 +51,16 @@ source:
50
51
transform :
51
52
- uses : add_field
52
53
with :
53
- expression : concat(firstname , ' ', lastname )
54
- field : fullname
55
- language : sql
54
+ expression : regex_replace(phone , '\+[0-9]+ (\([0-9]+\) )? ', '' )
55
+ field : localphone
56
+ language : jmespath
56
57
output :
57
58
- uses : redis.write
58
59
with :
59
60
connection : target
60
61
data_type : hash
61
62
key :
62
- expression : concat(['cust:', customerid ])
63
+ expression : concat(['cust:', uuid() ])
63
64
language : jmespath
64
65
` ` `
65
66
@@ -68,34 +69,33 @@ using [`redis-cli`]({{< relref "/develop/connect/cli" >}}), you would
68
69
see something like the following :
69
70
70
71
` ` `
71
- > hgetall cust:14
72
72
1) "customerid"
73
- 2) "14 "
73
+ 2) "27 "
74
74
3) "firstname"
75
- 4) "Mark "
75
+ 4) "Patrick "
76
76
5) "lastname"
77
- 6) "Philips "
78
- 7) "company"
79
- 8) "Telus"
80
- 9 ) "address "
81
- 10 ) "8210 111 ST NW "
77
+ 6) "Gray "
78
+ .
79
+ .
80
+ 17 ) "phone "
81
+ 18 ) "+1 (520) 622-4200 "
82
82
.
83
83
.
84
84
` ` `
85
85
86
- Using the job file above, the data also includes the new `fullname ` field :
86
+ Using the job file above, the data also includes the new `localphone ` field :
87
87
88
88
` ` `
89
- 1) "customerid"
90
- 2) "14 "
89
+ 1) "customerid"
90
+ 2) "27 "
91
91
3) "firstname"
92
- 4) "Mark "
92
+ 4) "Patrick "
93
93
5) "lastname"
94
- 6) "Philips "
94
+ 6) "Gray "
95
95
.
96
96
.
97
- 27 ) "fullname "
98
- 28 ) "Mark Philips "
97
+ 23 ) "localphone "
98
+ 24 ) "622-4200 "
99
99
` ` `
100
100
101
101
# # Add multiple fields
0 commit comments