Skip to content

MOLT 1.3.0 notes; --targetApplyQueueSize replicator flag #19988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 24, 2025
Merged

MOLT 1.3.0 notes; --targetApplyQueueSize replicator flag #19988

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f6f73c5
MOLT 1.3.0 notes; --targetApplyQueueSize replicator flag
taroface Jul 23, 2025
83f0322
address reviewer feedback
taroface Jul 24, 2025
dcf9b04
fix table
taroface Jul 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/current/_includes/molt/replicator-flags.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ The following flags are set with [`--replicator-flags`](#global-flags) and can b
| `--stagingMaxLifetime` | `DURATION` | The maximum lifetime of a database connection.<br><br>**Default:** `5m0s` |
| `--stagingMaxPoolSize` | `INT` | The maximum number of staging database connections.<br><br>**Default:** `128` |
| `--stagingSchema` | `STRING` | Name of the CockroachDB schema that stores replication metadata. **Required** each time [`--mode replication-only`](#replicate-changes) is rerun after being interrupted, as the schema contains a checkpoint table that enables replication to resume from the correct transaction. For details, refer to [Replicate changes](#replicate-changes).<br><br>**Default:** `_replicator.public` |
| `--targetApplyQueueSize` | `INT` | Size of the apply queue that buffers mutations before they are written to the target database. Larger values can improve throughput, but increase memory usage. This flag applies only to CockroachDB and PostgreSQL (`pglogical`) sources, and replaces the deprecated `--copierChannel` and `--stageCopierChannelSize` flags. |
| `--targetConn` | `STRING` | The target database's connection string. |
| `--targetIdleTime` | `DURATION` | Maximum lifetime of an idle connection.<br><br>**Default:** `1m0s` |
| `--targetJitterTime` | `DURATION` | The time over which to jitter database pool disconnections.<br><br>**Default:** `15s` |
Expand All @@ -39,7 +40,7 @@ The following flags are set with [`--replicator-flags`](#global-flags) and can b
| `--targetStatementCacheSize` | `INT` | The maximum number of prepared statements to retain.<br><br>**Default:** `128` |
| `--taskGracePeriod` | `DURATION` | How long to allow for task cleanup when recovering from errors.<br><br>**Default:** `1m0s` |
| `--timestampLimit` | `INT` | The maximum number of source timestamps to coalesce into a target transaction.<br><br>**Default:** `1000` |
| `--userscript` | `STRING` | The path to a TypeScript configuration script. For example, `--userscript 'script.ts'`. |
| `--userscript` | `STRING` | The path to a TypeScript configuration script. For example, `--userscript 'script.ts'`. |
| `-v`, `--verbose` | `COUNT` | Increase logging verbosity to `debug`; repeat for `trace`. |

##### PostgreSQL replication flags
Expand Down
18 changes: 18 additions & 0 deletions src/current/releases/molt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,24 @@ To download the latest MOLT Fetch/Verify binary:

{% include molt/molt-install.md %}

## July 24, 2025

MOLT Fetch/Verify 1.3.0 is [available](#installation).

- Oracle is now supported on Fetch Docker images and standalone binaries.
- Oracle is now supported on Replicator Docker images and standalone binaries.
- Azure Blob Storage is now supported as an intermediate data store for Fetch.
- Added a `--skip-pk-match` flag that lets you run the initial data load even when the source or target table is missing a primary key, or the keys do not match. When this flag is set, Fetch treats every table as keyless, disables sharding, and exports each table in a single batch, ignoring `export-concurrency` and `row-batch-size`. Querying the entire table at once may lead to high memory usage or long-running queries. Duplicate source rows are automatically removed during import when the target has primary key or unique constraints.
- Replication now uses checkpoint polling by default. This is because polling (which periodically queries the staging table for updates) performs comparably to checkpoint streaming (which uses an internal changefeed from the staging table to broadcast real-time updates). To use checkpoint streaming, set `--enableCheckpointStream`. `--disableCheckpointStream` is deprecated and should be removed from replication commands.
- Replaced the `--copierChannel` and `--stageCopierChannelSize` replication flags with a single `--targetApplyQueueSize` flag, which controls downstream apply throughput and memory usage. This feature applies only to CockroachDB and PostgreSQL (`pglogical`) sources.
- Added support for compressed changefeed payloads, improving performance of changefeed-to-Replicator communication. This only affects failback workflows from CockroachDB v25.2 and later.
- Improved apply-side performance by switching to a faster JSON decoder.
- Improved batch-accumulation performance by removing unnecessary sorting.

#### Bug fixes

- Fixed a bug where some error logs were not displayed when replicating to the target database.

## June 19, 2025

MOLT Fetch/Verify 1.2.7 is [available](#installation).
Expand Down
Loading