Skip to content

chore(deps): update dependency apache-airflow-providers-google to v16 #13446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore(deps): update dependency apache-airflow-providers-google to v16 #13446

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion composer/2022_airflow_summit/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# be sure to update the constraints file to match
# see https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#constraints-files
apache-airflow==2.6.3
apache-airflow-providers-google==10.2.0
apache-airflow-providers-google==16.0.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Updating apache-airflow-providers-google from 10.2.0 to 16.0.0 is a substantial upgrade, jumping from major version 10.x to 16.x. This carries a significant risk of introducing breaking changes, deprecations, or altered behavior in the Google Cloud operators and hooks.

Recommendations:

  • Thorough Changelog Review: Carefully examine the official apache-airflow-providers-google changelog for all versions between 10.2.0 and 16.0.0. Pay particular attention to any modifications affecting the operators and hooks used in your DAGs, such as:
    • DataprocCreateBatchOperator
    • BigQueryInsertJobOperator
    • GCSToBigQueryOperator
    • Authentication methods and connection configurations.
  • Comprehensive Testing: Before merging, conduct extensive testing of all DAGs that interact with Google Cloud services (especially data_analytics_dag.py and any others) in a staging or development environment. Verify that all tasks execute as expected and that data integrity is maintained.
  • Constraint File Synchronization: As noted in the comments within this file and in the general review summary, ensure the constraints.txt file is updated to apache-airflow-providers-google==16.0.0. This is crucial for consistent and reproducible Airflow environments.