Skip to content

Commit d80107f

Browse files
committed
Update documentation
1 parent ce53caa commit d80107f

File tree

1 file changed

+12
-26
lines changed
  • content/integrate/redis-data-integration/data-pipelines/prepare-dbs

1 file changed

+12
-26
lines changed

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/mongodb.md

Lines changed: 12 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -24,24 +24,13 @@ This guide describes the steps required to prepare a MongoDB database as a sourc
2424
- **Network access:** The RDI Collector must be able to connect to all MongoDB nodes in your deployment.
2525

2626
## 1. Configure Oplog Size
27-
Ensure the oplog is large enough to retain changes for the duration of your RDI pipeline's snapshot and streaming operations.
28-
29-
- Check oplog size:
30-
```javascript
31-
db.printReplicationInfo();
32-
```
33-
- Resize the oplog (run on the PRIMARY):
34-
```bash
35-
mongod --replSet <replSetName> --oplogSize <sizeInMB>
36-
```
27+
Ensure the oplog is large enough to retain changes for the duration of your RDI pipeline's snapshot and streaming operations.
28+
Follow the MongoDB [documentation](https://www.mongodb.com/docs/manual/tutorial/change-oplog-size/) to check and resize (if necessary) the oplog size.
3729

3830
## 2. Create a MongoDB User for RDI
3931
Create a user with the following roles on the source database:
40-
- read
4132
- readAnyDatabase
4233
- clusterMonitor
43-
- changeStream
44-
- readWrite (if you want to test writes)
4534

4635
Example:
4736
```javascript
@@ -50,10 +39,8 @@ db.createUser({
5039
user: "rdi_user",
5140
pwd: "rdi_password",
5241
roles: [
53-
{ role: "read", db: "your_db" },
5442
{ role: "readAnyDatabase", db: "admin" },
55-
{ role: "clusterMonitor", db: "admin" },
56-
{ role: "changeStream", db: "admin" }
43+
{ role: "clusterMonitor", db: "admin" }
5744
]
5845
});
5946
```
@@ -65,12 +52,16 @@ Example (Replica Set):
6552
```
6653
mongodb://${SOURCE_DB_USERNAME}:${SOURCE_DB_PASSWORD}@host1:27017,host2:27017,host3:27017/?replicaSet=rs0&authSource=admin
6754
```
55+
Example (Sharded Cluster):
56+
```
57+
mongodb://${SOURCE_DB_USERNAME}:${SOURCE_DB_PASSWORD}@host:30000
58+
```
6859
- For Atlas, adjust the connection string accordingly.
6960
- Set replicaSet and authSource as appropriate for your deployment.
7061

7162
## 4. Enable Change Streams and Pre/Post Images (Only if Using a Custom Key)
7263
Change Streams: Required only if you are using a custom key in your RDI pipeline. Change streams are available by default on replica sets, sharded clusters, and MongoDB Atlas.
73-
Pre/Post Images: If you use a custom key and want to capture full document changes on updates and deletes, enable pre- and post-images on collections as needed:
64+
Pre/Post Images: If your RDI pipeline uses a custom key, you must enable pre- and post-images on the relevant collections to capture the document state before and after updates or deletes. This allows RDI to access both the previous and updated versions of documents during change events, ensuring accurate synchronization.
7465
```javascript
7566
db.runCommand({
7667
collMod: "your_collection",
@@ -88,20 +79,15 @@ The root CA certificate for MongoDB Atlas must be added as a SOURCE_DB_CACERT se
8879

8980
Example connection string for Atlas:
9081
```
91-
mongodb+srv://rdi_user:rdi_password@cluster0.mongodb.net/?authSource=admin&tls=true
82+
mongodb+srv://${SOURCE_DB_USERNAME}:${SOURCE_DB_PASSWORD}@cluster0.mongodb.net/?authSource=admin&tls=true
9283
```
9384

9485
## 6. Network and Security
9586
- Ensure the RDI Collector can connect to all MongoDB nodes on the required ports (default: 27017, or as provided by Atlas).
9687
- If using TLS/SSL, provide the necessary certificates and connection options in the connection string.
9788

98-
## 7. Test the Connection
99-
You can test the connection using the RDI CLI or API:
100-
```bash
101-
redis-di pipelines validate-source --db-type mongodb \
102-
--connection-string "mongodb+srv://rdi_user:rdi_password@cluster0.mongodb.net/?authSource=admin&tls=true"
103-
```
104-
Or via the API `/collector/connection/validate` endpoint.
89+
## 7. Configuration is complete
90+
Once you have followed the steps above, your MongoDB database is ready for Debezium to use.
10591

10692
## Additional Resources
10793
- [MongoDB Replica Set Documentation](https://www.mongodb.com/docs/manual/replication/)
@@ -115,7 +101,7 @@ Or via the API `/collector/connection/validate` endpoint.
115101
| Requirement | Description |
116102
|---------------------|-----------------------------------------------------------------------------|
117103
| MongoDB Topology | Replica Set, Sharded Cluster, or MongoDB Atlas |
118-
| User Roles | read, readAnyDatabase, clusterMonitor, changeStream |
104+
| User Roles | readAnyDatabase, clusterMonitor |
119105
| Oplog | Sufficient size for snapshot and streaming |
120106
| Pre/Post Images | Enable on collections **only if using a custom key** |
121107
| Connection String | Must include all hosts, replicaSet (if applicable), authSource, credentials |

0 commit comments

Comments
 (0)