Code

Upgrade to Airflow 2.11.0#158

Code

Open

Lee-W wants to merge 31 commits intomasterpycontw/pycon-etl:masterfrom

upgrade-to-airflow-2pycontw/pycon-etl:upgrade-to-airflow-2Copy head branch name to clipboard

Commits

Commits on Jun 1, 2025

• feat: update airflow to 2.6.3 and Python to 3.10

  Show description for a20614a
  Lee-W

  committed
  a20614a

  Copy full SHA for a20614a

• fix: move airflow.db to sqlite/airflow.db

  Show description for 4322c15
  Lee-W

  committed
  4322c15

  Copy full SHA for 4322c15

• feat(airflow.cfg): migrate old airflow configuration to 2.6.3 version

  Lee-W

  committed
  222db94

  Copy full SHA for 222db94

• feat(airflow-config): add default webserver_config.py

  Lee-W

  committed
  2cce638

  Copy full SHA for 2cce638

• fix: add __init__.py for each module

  Lee-W

  committed
  6d568bd

  Copy full SHA for 6d568bd

• feat: upgrade to airflow 2.10.3

  Lee-W

  committed
  6651c09

  Copy full SHA for 6651c09

• feat: upgrade to airflow 2.11.0

  Lee-W

  committed
  5937af1

  Copy full SHA for 5937af1

• build(dev-dep): add types-requests>=2.31.0.1 and types-python-dateutil>=2.8.19.13

  Lee-W

  committed
  3e3eb90

  Copy full SHA for 3e3eb90

• fix(patch): add migration patch

  Show description for fe45eaf
  Lee-W

  committed
  fe45eaf

  Copy full SHA for fe45eaf

• build(Dockerfile): add patches to installed airflow

  Lee-W

  committed
  685cdb8

  Copy full SHA for 685cdb8

• build(docker-compose): add healthcheck to webserver

  Lee-W

  committed
  df3d38f

  Copy full SHA for df3d38f

• build(docker-compose): add airflow-init for enabling airflow migration

  Lee-W

  committed
  9f5af9a

  Copy full SHA for 9f5af9a

• build(Dockerfile): remove git installation as no longer needed

  Show description for 8233a16
  Lee-W

  committed
  8233a16

  Copy full SHA for 8233a16

• build(Dockerfile): remove gcc installation

  Lee-W

  committed
  90a23cf

  Copy full SHA for 90a23cf

• fix(dags): remove outdated airflow 1 syntax from "airflow-log-cleanup.py" and "airlfow-db-cleanup.py"

  Lee-W

  committed
  88ca57d

  Copy full SHA for 88ca57d

• style: format through ruff

  Lee-W

  committed
  99e1bdf

  Copy full SHA for 99e1bdf

• ci(github): update google-github-actions/auth to v2

  Lee-W

  committed
  20855e1

  Copy full SHA for 20855e1

• build: add mkdocs and mkdocs-material to doc dependency group

  Lee-W

  committed
  2d64b19

  Copy full SHA for 2d64b19

• docs: initialize mkdocs structure

  Lee-W

  committed
  7e0b2c5

  Copy full SHA for 7e0b2c5

• refactor(dags): update dags to use taskflow syntax whenever possible

  Lee-W

  committed
  295263f

  Copy full SHA for 295263f

• refactor(dag:DISCORD_CHORES_REMINDER): remove unnessary kwargs

  Lee-W

  committed
  125c31c

  Copy full SHA for 125c31c

• refactor(dag:DISCORD_FINANCE_REMINDER): improve typing and simplify logic

  Lee-W

  committed
  55d7e3d

  Copy full SHA for 55d7e3d

• refactor(dag:DISCORD_PROPOSAL_REMINDER_v3): remove unnessary kwargs

  Lee-W

  committed
  c43a664

  Copy full SHA for c43a664

• refactor(dag:KKTIX_DISCORD_BOT_FOR_TEAM_REGISTRATION): improve typing and simplify logic

  Lee-W

  committed
  f0a6fab

  Copy full SHA for f0a6fab

• docs(dag:TWITTER_POST_NOTIFICATION_BOT_V2): add airflow 3 upgrade note

  Lee-W

  committed
  266d04f

  Copy full SHA for 266d04f

• refactor(dag:FB_POST_INSIGHTS_V1): improve typing and simplify logic

  Lee-W

  committed
  f7df563

  Copy full SHA for f7df563

• refactor(dag:*post_insights): deduplicate common code with *PostsInsightsParser

  Lee-W

  committed
  18a47fa

  Copy full SHA for 18a47fa

• refactor(dag:*post_insights): extract bq_client as cached_property

  Lee-W

  committed
  1097e56

  Copy full SHA for 1097e56

• refactor(dag:*post_insights): extract common dump_to_bq_logic

  Lee-W

  committed
  12d0f57

  Copy full SHA for 12d0f57

• style: improve or temprorarily ignore mypy warnings

  Lee-W

  committed
  14fbd3f

  Copy full SHA for 14fbd3f

Commits on Jun 2, 2025

• build: add missing apache-airflow-providers-http

  Lee-W

  committed
  6c9a048

  Copy full SHA for 6c9a048