AWS Data Wrangler 0.0.13

Enhancements

• Documentation #54
• Add Athena.query() to receive the result as python primitives (Iterable[Dict[str, Any]) #55
• Add EMR.build_step()
• Add EMR.submit_steps()
• Add new parameters for EMR.create_cluster():
  • spark_log_level
  • spark_jars_path
  • spark_defaults
  • maximize_resource_allocation
  • steps
  • keep_cluster_alive_when_no_steps
  • termination_protected

Bugfixes

• Fix EMR.create_cluster() issue for single node clusters

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.12

Enhancements

• Add thousands and decimal parameters to Pandas.read_csv()

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.11

Enhancements

• Add security group parameters for EMR.create_cluster()

Bugfixes

• Fix pandas to redshift cast feature #48

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.10

New Functionalities

• Flatten nested PySpark DataFrames
• Create EMR cluster (For humans)
• Terminate EMR cluster
• Get EMR cluster state
• Submit EMR step (For humans)
• Get EMR step state

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.9

Bugfixes

• Fixing major and minor versions for all dependencies

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.8

Bugfixes

• Normalizing partitions and casting columns names #43

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.7

Enhancements

• Normalize Athena names (tables and columns) #39
• Removing Pandas duplicated columns #40
• Improving exception info #41
• Add Athena Workgroup #42

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.6

Enhancements

• Add compatibility with Windows O.S.

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.5

Enhancements

• Adding support for lists to Pandas #33
• Adding support for Date data type to load dataframes on Redshift #35
• Add support for timestamp with time zone for read from Athena #36

P.S. Lambda Layer's bundle and Glue's wheel/egg are available below. It's just upload and run!

AWS Data Wrangler 0.0.4

New Features

• Register Glue table from Dataframe stored on S3
• Load partitions on Athena/Glue table (repair table)

P.S. Lambda Layer's bundle and Glue's egg are available below. It's just upload to your account and run!