Skip to content

Commit 618af3d

Browse files
committed
adding an example for adding signal table and a new table to the pipeline (Oracle)
1 parent a285342 commit 618af3d

File tree

1 file changed

+67
-6
lines changed

1 file changed

+67
-6
lines changed

content/rdi/installation/adding-tables-to-existing-pipeline.md

Lines changed: 67 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Set up and use Debezium to add additional tables to an existing pip
55
weight: 80
66
alwaysopen: false
77
categories: ["redis-di"]
8-
aliases:
8+
aliases:
99
---
1010

1111
If you want to add a new table to a pipeline that is already in streaming (CDC) mode, you can do so without resetting Debezium Server and executing a new full snapshot. In Debezium, this is called incremental snapshot and it is performed using a table on the source database as the interface with the Debezium connector.
@@ -114,14 +114,75 @@ The data-collections array lists tables by their fully-qualified names, using th
114114

115115
#### Signaling Table Columns
116116

117-
| Column | Description |
118-
| ------ | --------------------------------------------------------------------------------- |
119-
| id | An arbitrary string that is assigned as the identifier for the signal request. |
120-
| type | The type of signal to send. |
121-
| data | An array of table names to include in the snapshot. |
117+
| Column | Description |
118+
| ------ | ------------------------------------------------------------------------------ |
119+
| id | An arbitrary string that is assigned as the identifier for the signal request. |
120+
| type | The type of signal to send. |
121+
| data | An array of table names to include in the snapshot. |
122122

123123
## SQL Server: `sp_cdc_enable_table` Stored Procedure Arguments
124124

125125
- `@source_name` - Specifies the name of the table that you want to capture.
126126
- `@role_name` - Specifies a role MyRole to which you can add users to whom you want to grant SELECT permission on the captured columns of the source table. Users in the sysadmin or db_owner role also have access to the specified change tables. Set the value of @role_name to NULL, to allow only members in the sysadmin or db_owner to have full access to captured information.
127127
- `@filegroup_name` - Specifies the filegroup where SQL Server places the change table for the captured table. The named filegroup must be already exist. It is best not to locate change tables in the same filegroup that you use for source tables.
128+
129+
## Example for Adding a Signaling Table for Oracle Database
130+
131+
1. Creating a signaling table `DEBEZIUM_SIGNAL`:
132+
133+
```sql
134+
CREATE TABLE DEBEZIUM_SIGNAL
135+
(
136+
id VARCHAR(42) PRIMARY KEY,
137+
type VARCHAR(32) NOT NULL,
138+
data VARCHAR(2048) NULL
139+
);
140+
```
141+
142+
2. Adding the property `debezium.source.signal.data.collection` to the `application.properties` file:
143+
144+
```properties
145+
debezium.source.signal.data.collection=ORCLPDB1.C##DBZUSER.DEBEZIUM_SIGNAL
146+
```
147+
148+
> Note: The property `debezium.source.signal.data.collection` should be set to the fully qualified name of the table. In `Oracle`, the fully qualified name includes the schema name `C##DBZUSER` and the database name `ORCLPDB1`.
149+
150+
3. Enable supplemental logging for the `DEBEZIUM_SIGNAL` table:
151+
152+
```
153+
ALTER table c##dbzuser.DEBEZIUM_SIGNAL add SUPPLEMENTAL LOG DATA(ALL) COLUMNS;
154+
```
155+
156+
> Note: If the supplemental logging is enabled for the entire database you can skip this step.
157+
158+
4. Restart the Debezium Server
159+
160+
## Example for Adding the EMP table to the pipeline
161+
162+
1. Add the `EMP` table to the `debezium.source.table.include.list` property in the `application.properties` file:
163+
164+
```properties
165+
debezium.source.table.include.list=C##DBZUSER.EMP
166+
```
167+
168+
2. Enable supplemental logging for the `EMP`` table:
169+
170+
To enable supplemental logging for all the table columns:
171+
172+
```sql
173+
ALTER Table c##dbzuser.EMP add SUPPLEMENTAL LOG DATA(ALL)
174+
COLUMNS
175+
```
176+
177+
3. Restart the `Debezium Server`
178+
179+
4. To trigger the `incremental snapshot` for the `EMP table, run:
180+
181+
```sql
182+
INSERT INTO c##dbzuser.DEBEZIUM_SIGNAL ds (id, type, data)
183+
VALUES ('1', 'execute-snapshot', '{"data-collections":["ORCLPDB1.C##DBZUSER.EMP"],"type":"incremental"}');
184+
```
185+
186+
> Note: The column `id` is a unique string in the DEBEZIUM_SIGNAL table
187+
188+
5. The `EMP` table will be added to the pipeline, and irs keys will be stored in the RDI bdb with no need to run `redis-di reset`

0 commit comments

Comments
 (0)