0.13.1 (2020-03-02)

Bump ansible to 2.7.16

FastSync MySQL

• Handle reserved words in column names in FastSync from MySQL
• Fixed issue when parallelism and parallelism_max parameters were not used in tap YAML files

Tap Postgres

• Bump tap-postgres to 1.4.1
  • Remove unused timestamps in log

0.13.0 (2020-02-26)

• Structured logs in Pipelinewise, FastSync and majority of plugins.
• Include a logging config file in Pipelinewise repository and package.
• Ability to provide a custom logging config by setting the env variable LOGGING_CONF_FILE to be the path to the .conf file.

0.12.4 (2020-02-19)

Merge pull request #325 from transferwise/release-0.12.4

[RELEASE] Bump to 0.12.4 and add changelog

0.12.3 (2020-02-19)

Merge pull request #323 from transferwise/bump-patch

[RELEASE] Bump version

0.12.2 (2020-02-03)

Transform field

• Bump pipelinewise-transform-field to 1.1.2
  • Make validation turned off by default.

0.12.1 (2020-01-31)

• FastSync: Changed the default /tmp folder for snowflake encryption

Target Snowflake

• Bump pipelinewise-target-snowflake to 1.4.1
  • Changed the default /tmp folder for encryption

0.12.0 (2020-01-21)

• FastSync: Support BINARY and VARBINARY column types from MySQL sources
• FastSync: Fixed an issue when MASK-HIDDEN type of transformations were not applied in Snowflake targets
• Write temporary files to ~/.pipelinewise/tmp directory
• Add stop_tap command
• Fixed an issue when post import Primary Keys check was not working correctly
• Fixed an issue when discover_tap command sometimes was failing

Tap MySQL

• Bump pipelinewise-tap-mysql to 1.1.3
  • Support to extract BINARY and VARBINARY column types
  • Improved performance of reading data from MySQL binary log
  • Increase default session wait_timeout to 28800
  • Increase default session innodb_lock_wait_timeout to 3600

Tap S3 CSV

• Bump pipelinewise-tap-s3-csv to 1.0.7
  • Improved column type guesser

Tap Kafka

• Bump pipelinewise-tap-kafka to 2.0.0
  • Rewamp output schema, export the consumed JSON messages from Kafka topics to fixed columns
  • Disable data flattening

Target Snowflake

• Bump pipelinewise-target-snowflake to 1.3.0
  • Load binary data into Snowflake BINARY column types
  • Adjust timestamps from taps automatically to the max allowed 9999-12-31 23:59:59 when it's required
  • Add validate_record optional parameter and default to False
  • Add temp_dir optional parameter to overwrite system defaults

Update target-snowflake connector

Merge pull request #287 from transferwise/AP-369

0.11.4: update target-snowflake component version to latest 1.1.7

Update tap-mysql connector

Merge pull request #286 from transferwise/Samira-El-patch-1

Use latest pipelinewise-tap-mysql

0.11.1 (2019-11-28)

• FastSync: Add fastsync support from S3-CSV to Snowflake
• Add post import checks to detect tables with no primary key early
• Add optional --connectors to the install script to install taps and targets selectively

Tap Zendesk

• Forked singer connector to pipelinewise-tap-zendesk==1.0.0
  • Improved performance by getting data from Zendesk API in parallel

Tap Postgres

• Bump pipelinewise-tap-postgres to 1.3.0
  • Add max_run_seconds configurable option
  • Add break_at_end_lsn configurable option
  • Only send feedback when lsn_comitted has increased

Tap Snowflake

• Bump pipelinewise-tap-snowflake to 1.0.5
  • Bump snowflake-connector-python to 2.0.4

Tap Kafka

• Bump pipelinewise-tap-kafka to 1.0.2
  • Add encoding configurable option

Target Redshift

• Bump pipelinewise-target-redshift to 1.1.0
  • Emit new state message as soon as data flushed to Redshift
  • Add flush_all_streams option
  • Add max_parallelism option