v0.6.5

What's Changed

• Json transformer support by @agedemenli in #389
• [Replication] Add support for generated columns with postgres writer by @eminano in #391
• [Snapshot] Wait for all tables to finish processing data by @eminano in #395
• Fix transformers panic by @eminano in #394

Full Changelog: v0.6.4...v0.6.5

v0.6.4

What's Changed

• Add strict validation for all tables by @agedemenli in #364
• [Snapshot] Add support for providing pg_dump role by @eminano in #371
• [Snapshot] Do not copy generated columns by @eminano in #374
• [Snapshot] Disable target database creation by default by @eminano in #375
• Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc from 1.35.0 to 1.36.0 by @dependabot in #376
• Bump github.com/jackc/pgx/v5 from 5.7.4 to 5.7.5 by @dependabot in #377
• Bump github.com/labstack/echo/v4 from 4.13.3 to 4.13.4 by @dependabot in #378
• Bump github.com/nucleuscloud/neosync from 0.5.37 to 0.5.39 by @dependabot in #379
• Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc from 1.35.0 to 1.36.0 by @dependabot in #380
• [Postgres Writer] Fix updates/deletes when no PK by @eminano in #382
• [Postgres Writer] Update batch defaults when bulk enabled by @eminano in #384

Full Changelog: v0.6.3...v0.6.4

v0.6.3

Changelog

• 1af00d1 Do not dump if there's no sequences
• b366c76 Merge pull request #370 from xataio/bugfix-snapshot-with-no-sequences

What's Changed

• [Snapshot] Do not dump sequence values if there's no sequences by @eminano in #370

Full Changelog: v0.6.2...v0.6.3

v0.6.2

Changelog

• 662c945 Add CopyFrom to pg lib transaction
• dacd2ea Add create option to pg_dump/pg_restore
• a9a2dcd Add create option to schema snapshot generator
• 712dde8 Merge branch 'main' into fix-disable-triggers
• 16644ab Merge branch 'main' into fix-disable-triggers
• b97a31b Merge pull request #367 from xataio/snapshot-create-database-if-not-exists
• dc5124b Merge pull request #368 from xataio/fix-disable-triggers
• c5c89a8 Merge pull request #369 from xataio/snapshot-sequence-values
• c53db6c Set replication role per session
• d123e62 Set val for dump sequences

What's Changed

• [Snapshot] Add option to create database if not exists by @eminano in #367
• [Snapshot] Set values for dump sequences by @eminano in #369
• Fix disable triggers by @eminano in #368

Full Changelog: v0.6.1...v0.6.2

v0.6.1

Changelog

• ed363f6 Merge pull request #366 from xataio/fix-default-to-bulk-ingest-on-snapshot-env
• ff9b09e Set bulk insert to true for snapshot cmd

What's Changed

• [Snapshots] Set bulk insert to true for snapshot cmd by default by @eminano in #366

Full Changelog: v0.6.0...v0.6.1

v0.6.0

⚡ Highlights

🔒 Last name and full name transformers
📝 Template transformer
🔭 Observability improvements
🚀 Snapshot performance improvements

What's Changed

• [CLI] Integrate instrumentation by @eminano in #327
• Add neosync lastname transformer by @agedemenli in #328
• [Transformers] Add instrumentation by @eminano in #329
• Add neosync fullname transformer by @agedemenli in #333
• [Webhook Notifier] Do not close notify channel on send error by @eminano in #330
• Add instrumentation to postgres lib by @eminano in #334
• Improve processor instrumentation by @eminano in #335
• Make schemalog store cache concurrency safe by @eminano in #336
• [Snapshots] Add instrumentation by @eminano in #337
• [Postgres Batch Writer] Add instrumentation by @eminano in #338
• Add template transformer by @agedemenli in #339
• Template transformer: Add greenmask toolkit functions by @agedemenli in #340
• Update docs to add template support by @agedemenli in #341
• Add basic profiling support to CLI by @eminano in #342
• Improve err msg for data type incompatibility in transformer validation by @agedemenli in #345
• Allow empty transformer list config by @agedemenli in #346
• Add sprig template functions by @agedemenli in #348
• Phone number dynamic country code by @agedemenli in #349
• Refactor all transformers to introduce definitions by @agedemenli in #350
• Add COPY FROM support to internal postgres library by @eminano in #352
• Add runtime metrics to otel provider by @eminano in #353
• Update docker setup by @eminano in #354
• Bump go.opentelemetry.io/otel/trace from 1.35.0 to 1.36.0 by @dependabot in #357
• Bump go.opentelemetry.io/contrib/instrumentation/runtime from 0.60.0 to 0.61.0 by @dependabot in #359
• Bump github.com/segmentio/kafka-go from 0.4.47 to 0.4.48 by @dependabot in #360
• [Postgres Processor] Add bulk ingest writer by @eminano in #355
• [Snapshots] Support negative values for table page count by @eminano in #363
• [Snapshots] Create indices/constraints after data dump by @eminano in #362
• [Snapshots] Use exclude schemas on schema pg_dump by @eminano in #356
• [Instrumentation] Add pgstream signoz dashboard by @eminano in #365

Full Changelog: v0.5.0...v0.6.0

v0.5.0

⚡ Highlights

🔒 Masking, phone number and literal transformers added
🎭 Dynamic parameter support for custom transformers
🎛️ YAML configuration support
✅ Transformation rules validation
⏭️ Table filtering support
💻 CLI command refactoring

What's Changed

• Add masking support by @agedemenli in #217
• [Snapshots] Add clean option to schema snapshot by @eminano in #218
• Bump github.com/elastic/go-elasticsearch/v8 from 8.14.0 to 8.17.1 by @dependabot in #220
• Bump github.com/rs/xid from 1.5.0 to 1.6.0 by @dependabot in #221
• Bump github.com/testcontainers/testcontainers-go/modules/elasticsearch from 0.33.0 to 0.36.0 by @dependabot in #222
• Bump github.com/jackc/pgx/v5 from 5.7.2 to 5.7.4 by @dependabot in #223
• Bump github.com/rs/zerolog from 1.33.0 to 1.34.0 by @dependabot in #224
• [Snapshots] Support repeatable pgdump snapshots without clean mode by @eminano in #219
• Add phone number transformer by @tsg in #232
• [CLI] Bug fixes by @eminano in #229
• [Snapshot] pg_dump snapshot fixes by @eminano in #230
• [Postgres Processor] Support custom type mapping by @eminano in #236
• Add random and deterministic generators by @agedemenli in #237
• Rewrite phone number transformer using generators by @agedemenli in #239
• [Transformers] Support dynamic parameters by @eminano in #240
• [Batch sender] Graceful shutdown by @eminano in #248
• feat(cli-definition): generate CLI definition as Json by @divyenduz in #250
• Bump golang.org/x/sync from 0.12.0 to 0.13.0 by @dependabot in #251
• Bump github.com/spf13/viper from 1.19.0 to 1.20.1 by @dependabot in #253
• Bump github.com/jonboulle/clockwork from 0.4.0 to 0.5.0 by @dependabot in #254
• Bump github.com/pterm/pterm from 0.12.79 to 0.12.80 by @dependabot in #255
• Bump github.com/nucleuscloud/neosync from 0.5.15 to 0.5.35 by @dependabot in #252
• Add yaml config support by @eminano in #256
• Validate pg column types with transformer types by @agedemenli in #249
• Add snapshot recorder url to yaml config by @eminano in #263
• Validate transformer parameters by @agedemenli in #264
• Add option to repeat snapshots when using snapshot recorder by @eminano in #265
• Improve error message unsupported transformer config to include name by @agedemenli in #271
• Add custom masking support by @agedemenli in #267
• Reset config file after reading transformation file by @eminano in #272
• Bump github.com/nucleuscloud/neosync from 0.5.36 to 0.5.37 by @dependabot in #276
• Read config case sensitive, using yaml unmarshal by @agedemenli in #279
• Allow to provide env variables without config by @eminano in #278
• Add table filtering support by @eminano in #275
• Merge wildcard tables in filter by @eminano in #287
• [Snapshots] Fix repeatable snapshots by @eminano in #284
• Bump github.com/elastic/go-elasticsearch/v8 from 8.17.1 to 8.18.0 by @dependabot in #290
• Add table name and pg type oid to transformer data type error message by @agedemenli in #289
• Use connection pool in transformer validator by @eminano in #293
• Close rows immediately to release connection by @eminano in #294
• Consolidate snapshot env vars by @eminano in #292
• Add noop transformers support and Strict validation mode by @agedemenli in #288
• Update docs for transformer validation by @agedemenli in #295
• [CLI] Split run and snapshot commands by @eminano in #291
• Update kafka config by @eminano in #297
• [CLI] Refactor and add command flags by @eminano in #298
• Update docs by @eminano in #299
• Bump github.com/golang-migrate/migrate/v4 from 4.18.2 to 4.18.3 by @dependabot in #300
• Bump github.com/testcontainers/testcontainers-go/modules/postgres from 0.36.0 to 0.37.0 by @dependabot in #301
• Bump github.com/testcontainers/testcontainers-go/modules/kafka from 0.36.0 to 0.37.0 by @dependabot in #302
• Bump github.com/testcontainers/testcontainers-go/modules/elasticsearch from 0.36.0 to 0.37.0 by @dependabot in #303
• Use the right flags for init/teardown binding by @eminano in #306
• Error if table_transformers not provided, when there's a transformati… by @agedemenli in #309
• UUID Transformer: Do not parse string into uuid by @agedemenli in #313
• Add make build with version by @eminano in #314
• Fix syntax error with ddl adapter by @eminano in #312
• Improve errors by @eminano in #315
• Use current commit in version by @eminano in #316
• [CLI] Update defaults on snapshot cmd by @eminano in #317
• Bump golang.org/x/sync from 0.13.0 to 0.14.0 by @dependabot in #320
• Bump github.com/testcontainers/testcontainers-go/modules/opensearch from 0.36.0 to 0.37.0 by @dependabot in #321
• Add literal string transformer by @agedemenli in #318
• Update yaml examples by @naveen246 in #322
• [CLI] Add status command by @eminano in #307
• Update docs for literal string transformer by @agedemenli in #324
• Update docs for phone number transformer by @agedemenli in #325
• Update docs by @eminano in #326

New Contributors

• @divyenduz made their first contribution in #250
• @naveen246 made their first contribution in #322

Full Changelog: v0.4.1...v0.5.0

v0.4.1

⚡ Highlights

🐛 Bug fixes
Dependency updates
🗒️ Documentation updates

What's Changed

• Fix choice return type by @agedemenli in #182
• [Transformers] Add support to find array parameters by @eminano in #186
• [Transformers] Make generator type a greenmask parameter by @eminano in #187
• Update transformer docs by @agedemenli in #181
• Polish docs by @eminano in #191
• [Snapshots] Do not create public schema by @eminano in #189
• Update snapshot tutorial by @eminano in #190
• Fix excluded_domains param for emails by @agedemenli in #195
• [Postgres Processor] Improve batch writer logs by @eminano in #192
• Add gif to readme by @eminano in #194
• [Snapshots] Fix data snapshot concurrency by @eminano in #200
• Add integration test for transformers by @agedemenli in #202
• Bump github.com/expr-lang/expr from 1.16.9 to 1.17.0 by @dependabot in #205
• Bump golang.org/x/net from 0.34.0 to 0.36.0 by @dependabot in #206
• Bump go.opentelemetry.io/otel/trace from 1.34.0 to 1.35.0 by @dependabot in #207
• [Snapshots] Fix DDL replication with pg_dump/pg_restore schema snapshot by @eminano in #201
• [Batch sender] Do not close msg channel on send error by @eminano in #212
• Catch value too long error by @agedemenli in #213
• Update golangci-lint by @eminano in #214
• Bump github.com/spf13/cobra from 1.8.1 to 1.9.1 by @dependabot in #208
• Bump github.com/bytedance/sonic from 1.12.3 to 1.13.2 by @dependabot in #209
• Bump github.com/labstack/echo/v4 from 4.12.0 to 4.13.3 by @dependabot in #211
• Bump golang.org/x/sync from 0.11.0 to 0.12.0 by @dependabot in #215
• Bump github.com/golang-migrate/migrate/v4 from 4.18.1 to 4.18.2 by @dependabot in #216

Full Changelog: v0.4.0...v0.4.1

v0.4.0

⚡ Highlights

🐘 PostgreSQL processor added
📷 Snapshot only mode support
🔒 Column transformation support
🗒️ Improved documentation

What's Changed

• Support int32 as id column by @agedemenli in #109
• Fix validation error parsing on opensearch by @eminano in #110
• DRY processor batch sender by @eminano in #119
• Rename translator to injector by @eminano in #124
• Add Postgres output with DML support by @eminano in #125
• Add enriched schema diff computation by @eminano in #131
• Add pg writer ddl support by @eminano in #137
• Add pg2pg integration tests by @eminano in #138
• Fix flaky pg2pg integration tests by @eminano in #142
• Add transformers package by @eminano in #140
• Add transformer processor by @eminano in #143
• Add pg_dump/pg_restore to internal library by @eminano in #144
• Restructure snapshot generator pkg by @eminano in #147
• Treat snapshot errors as strings for recording by @eminano in #148
• Add pg_dump/pg_restore schema snapshot generator by @eminano in #149
• Use pg_dump/pg_restore snapshot generator in pg listener by @eminano in #152
• Add pg2pg snapshot integration tests by @eminano in #153
• Decouple snapshot from wal listener by @eminano in #156
• Add snapshot listener by @eminano in #157
• Add gm integer transformer by @agedemenli in #154
• Add gm float transformer by @agedemenli in #155
• Add UUID gm transformer by @agedemenli in #158
• Add gm boolean transformer by @agedemenli in #159
• Add choice gm transformer by @agedemenli in #160
• Add unix timestamp gm transformer by @agedemenli in #162
• Add date transformer using greenmasks timestamp by @agedemenli in #163
• Add utc timestamp gm transformer by @agedemenli in #161
• Fix env file variable name by @eminano in #167
• Add ns email transformer by @agedemenli in #164
• Update README.md by @jacokok in #169
• [PostgreSQL processor] Handle errors gracefully by @eminano in #174
• Do not use int64 as yaml parameters by @agedemenli in #175
• [PostgreSQL processor] Add handling of syntax error by @eminano in #179
• [Transformer processor] Do not transform nil values and treat errors as dataloss by @eminano in #180
• Improve docs by @eminano in #168

New Contributors

• @agedemenli made their first contribution in #109
• @jacokok made their first contribution in #169

Full Changelog: v0.3.1...v0.4.0

v0.3.1

What's Changed

• Fix processing bug by @eminano in #107

Full Changelog: v0.3.0...v0.3.1