|
| 1 | +--- |
| 2 | +Title: Redis Data Integration release notes 1.8.0 (May 2025) |
| 3 | +alwaysopen: false |
| 4 | +categories: |
| 5 | +- docs |
| 6 | +- operate |
| 7 | +- rs |
| 8 | +description: | |
| 9 | + Enhanced RDI operator for better pipeline orchestration, resiliency, observability and flexibility; |
| 10 | + External collector support; |
| 11 | + Labels and annotations to RDI data plane pods; |
| 12 | + Custom Debezium image; |
| 13 | + Calculated TTL for target database. |
| 14 | +linkTitle: 1.8.0 (May 2025) |
| 15 | +toc: 'true' |
| 16 | +weight: 984 |
| 17 | +--- |
| 18 | + |
| 19 | +{{< note >}}This minor release replaces the 1.6.7 release.{{< /note >}} |
| 20 | + |
| 21 | +RDI’s mission is to help Redis customers sync Redis Enterprise with live data from their slow disk-based databases to: |
| 22 | + |
| 23 | +- Meet the required speed and scale of read queries and provide an excellent and predictable user experience. |
| 24 | +- Save resources and time when building pipelines and coding data transformations. |
| 25 | +- Reduce the total cost of ownership by saving money on expensive database read replicas. |
| 26 | + |
| 27 | +RDI keeps the Redis cache up to date with changes in the primary database, using a [_Change Data Capture (CDC)_](https://en.wikipedia.org/wiki/Change_data_capture) mechanism. |
| 28 | +It also lets you _transform_ the data from relational tables into convenient and fast data structures that match your app's requirements. You specify the transformations using a configuration system, so no coding is required. |
| 29 | + |
| 30 | +## Headlines |
| 31 | + |
| 32 | +- Enhanced RDI operator for better pipeline orchestration, resiliency, observability, and flexibility. It |
| 33 | + will also enable many new features in the near future. |
| 34 | +- You can now use an external collector that is not managed by RDI but writes into RDI streams |
| 35 | + (Debezium compatible). |
| 36 | +- You can now add labels and annotations to RDI data plane pods, for example to control service |
| 37 | + mesh features. |
| 38 | +- RDI now uses a custom image of Debezium (based on `3.0.8.Final`) to address known vulnerabilities. |
| 39 | +- Added support for calculated TTL for target database keys via `expire` expressions. |
| 40 | + |
| 41 | +## Detailed changes |
| 42 | + |
| 43 | +### Helm chart changes |
| 44 | + |
| 45 | +- All collector and processor values that were previously under `collector`, `collectorSourceMetricsExporter`, and `processor` have been moved to `operator.dataPlane.collector` and `operator.dataPlane.processor`. |
| 46 | +- `global.collectorApiEnabled` has been moved to `operator.dataPlane.collectorApi.enabled`, and is now a boolean value (`true` or `false`), not `"0"` or `"1"`. |
| 47 | +- `api.authEnabled` is also now a boolean value, not `"0"` or `"1"`. |
| 48 | +- The following values have been deprecated: `rdiMetricsExporter.service.protocol`, `rdiMetricsExporter.service.port`, `rdiMetricsExporter.serviceMonitor.path`, `api.service.name` |
| 49 | +- You can now add custom labels and annotations to all RDI components. |
| 50 | +- You can now disable the creation of the RDI system secrets. |
| 51 | + |
| 52 | +### Operator Improvements |
| 53 | + |
| 54 | +The RDI operator has been significantly enhanced in the following areas: |
| 55 | + |
| 56 | +- **Resilience**: The operator now always maintains the desired pipeline state. Manual changes or random disruptions are reverted automatically. |
| 57 | +- **Automatic recovery**: When a configuration issue is resolved, the entire pipeline starts automatically, eliminating the need for manual redeployment. |
| 58 | +- **Consistency**: A pipeline that has been stopped with `stop` will remain stopped after `deploy` or `reset`, until explicitly started again. |
| 59 | +- **Enhanced configuration**: You can now configure data plane components in ways that were previously not supported, such as adding labels and annotations. |
| 60 | +- **External collector support**: No collector resources are created for sources of type `external`. |
| 61 | +- **Enhanced troubleshooting**: You can now gain extra insight into the pipeline state by examining the `Pipeline` and `PipelineRelease` custom K8s resources. |
| 62 | + |
| 63 | +### Other Features, Improvements and Enhancements |
| 64 | + |
| 65 | +- Added `expire` expression for target output in transformation jobs. |
| 66 | +- Addressed security vulnerabilities: TLS certificate hostname verification is now ON by default. |
| 67 | +- Improved Helm default values while preserving `values.yaml` formatting. |
| 68 | +- Enhanced Helm values and templates for better configuration. |
| 69 | +- Added a script to create or update secrets when using Helm (`rdi-secret.sh` in the Helm zip file). |
| 70 | +- Improved validation schema and ensured backward compatibility. |
| 71 | +- Fixed compatibility issues with newer versions of `requests` and `urllib3`. |
| 72 | +- Improved error messages for JSON schema validation. |
| 73 | +- Improved PostgreSQL documentation for mTLS. |
| 74 | +- Added timestamps to the `status` command. |
| 75 | +- Fixed issues with `primary_key` and `unique_constraint` attributes in Oracle metadata. |
| 76 | +- Added `capture.mode` to MongoDB scaffolding. |
| 77 | +- Improved Helm TLS setup for RDI database connections. |
| 78 | +- Enhanced error handling and validation for transformation jobs. |
| 79 | +- Improved documentation for supported platforms and configurations. |
| 80 | + |
| 81 | +### Fixes |
| 82 | + |
| 83 | +- Fixed HTTP 500 error when querying columns with tables parameter. |
| 84 | +- Improved Helm TLS setup for RDI database connections. |
| 85 | +- Fixed keystore overwrite when using mTLS on both source and RDI DBs in the collector. |
| 86 | + |
| 87 | +## Limitations |
| 88 | + |
| 89 | +RDI can write data to a Redis Active-Active database. However, it doesn't support writing data to two or more Active-Active replicas. Writing data from RDI to several Active-Active replicas could easily harm data integrity as RDI is not synchronous with the source database commits. |
0 commit comments