Spark On K8S 0.12.1

Bug Fixes:

• fix: workaround for TLS handshake error when reading pod logs by @hussein-awala in #135

Misc:

• chore: update dependencies by @hussein-awala in #136
• fix: update mkdocs configs to fix it by @hussein-awala in #137
• feat(docs): docs versioning with mike by @hussein-awala in #138

Full Changelog: 0.12.0...0.12.1

Spark On K8S 0.12.0

Bug Fixes:

• fix(airflow): adopt job only from the same dag_run_id/exection_date by @hussein-awala in #128

New Features:

• feat(api): add 2 configs for delete and kill buttons in the webserver by @hussein-awala in #132
• feat(api): add a config to control and disable refresh interval in apps list by @hussein-awala in #133

Misc:

• docs: update docs link for pip package by @hussein-awala in #129
• chore: update dependencies by @hussein-awala in #130

Full Changelog: 0.11.0...0.12.0

Spark On K8S 0.11.0

What's Changed

Significant Changes

• chore: remove support for Python 3.8 by @hussein-awala in #96

Bug Fixes:

• fix(chart): add missing config to spark-history deployment by @hussein-awala in #104
• fix(doc): fix failed mkdocs and remove warnings by @hussein-awala in #111
• fix(webserver): fix broken apps endpoint by @hussein-awala in #117
• fix(python-client): truncate driver pod name and fix app name/id truncation by @hussein-awala in #119

New Features:

• feat(airflow): add on_finish_action to decide what to do when the job finishes by @hussein-awala in #100
• feat(python-client): add more config for dynamic allocation by @hussein-awala in #102
• feat(cli): add more config for dynamic allocation by @hussein-awala in #103
• feat(airflow): create SparkSqlOnK8SOperator to execute Spark SQL commands by @hussein-awala in #109
• feat(python-client): Support for init containers in driver pod by @hussein-awala in #121
• feat(python-client): Support for host aliases in driver pod by @hussein-awala in #122

Misc

• chore(airflow): create an enum for app_waiter by @hussein-awala in #101
• chore(spark-history): update spark-history image to jdk 17 and spark 3.5.3 by @hussein-awala in #105
• chore: update spark version in java example by @hussein-awala in #107
• chore: user StrEnum instead of str, Enum by @hussein-awala in #108
• chore(doc): add some examples for SparkSqlOnK8SOperator operator by @hussein-awala in #112
• chore: add sqlfluff to lint spark sql queries by @hussein-awala in #113
• chore(doc): add some examples for driver configuration by @hussein-awala in #125

Full Changelog: 0.10.1...0.11.0

Spark On K8S 0.10.1

What's Changed

• fix(airflow): load namespace and pod_name from trigger event by @hussein-awala in #89

Full Changelog: 0.10.0...0.10.1

Spark On K8S 0.10.0

What's Changed

• security: bump zipp to avoid CVE-2024-5569 by @hussein-awala in #81
• security: bump apache-airflow to avoid CVE-2024-39863 and CVE-2024-39877 by @hussein-awala in #82
• security: update setuptools to avoid CVE-2024-6345 by @hussein-awala in #83
• fix: propagate service account + test by @rivamarco in #88
• feat(python-client, airflow): support startup_timeout for pending state waiting by @hussein-awala in #84

New Contributors

• @rivamarco made their first contribution in #88

Full Changelog: 0.9.1...0.10.0

Spark On K8S 0.9.1

What's Changed

• fix(airflow): fix xcom pull in _try_to_adopt_job by @hussein-awala in #80

Full Changelog: 0.9.0...0.9.1

Spark On K8S 0.9.0

What's Changed

• feat(api): support pagination on list_apps route by @hussein-awala in #77
• feat: add support for packages param by @hussein-awala in #79

Full Changelog: 0.8.0...0.9.0

Spark On K8S 0.8.0

What's Changed

• feat(python-client): support creating and mounting an ephemeral configmap to the driver by @hussein-awala in #74
• feat(airflow): support driver_ephemeral_configmaps_volumes by @hussein-awala in #75
• security: update urllib3 to avoid CVE-2024-37891 by @hussein-awala in #69
• security: bump apache-airflow to avoid CVE-2024-32077 by @hussein-awala in #70
• security: update requests to avoid CVE-2024-35195 by @hussein-awala in #71
• security: bump certifi to avoid CVE-2024-39689 by @hussein-awala in #76
• docs: update python example to explain how to use custom modules by @hussein-awala in #73

Full Changelog: 0.7.2...0.8.0

Spark On K8S 0.7.2

What's Changed

• fix(airflow): fix spark_history_ui_link by removing -driver suffix by @hussein-awala in #68

Full Changelog: 0.7.1...0.7.2

Spark On K8S 0.7.1

What's Changed

• fix(airflow): configure provider metadata to make the extra-link discoverable by @hussein-awala in #60
• security: bump Jinja2 to avoid CVE-2024-34064 by @hussein-awala in #61

Full Changelog: 0.7.0...0.7.1