Skip to content

Move webserver expose_hostname config to fab #50269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

pierrejeambrun
Copy link
Member

Part of: #49896

Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here - what if somebody uses a new FAB with old core?

@potiuk
Copy link
Member

potiuk commented May 12, 2025

Yes. I think such changes in Airflow should be minor version upgrades -> 3.1.0 should be minimum version

Also - we have to be quite careful when making such change. We should at least have a mental model and work out scenarios where various versions and combos of airlfow + fab would be installed (or prevent them from being installed together if those combos are not good).

I don't think we have a realy good guide for what happens when you move config from Airflow to provider like thiat. The ConfigChange we have currently implemented does not serve well case where configurations are moved between two distributions, It only works if ConfigChange is moved within Airflow or within distribution.

Assume we add this in 2.1.0 FAB (not 2.0.2) - moving configuration like that is for me sign of "minor release" at least.

Two easy cases are :

  • FAB 2.0.1 + Airflow 3.0.1 (only config in Airlfow works, no-one suggests to move it)

  • FAB 2.1.0 + Airflow 3.1.0 - config should be in FAB, if someone still has it in airflow it is going to work wth deprecation.

  • More interesting is: 2.0.1 FAB and 3.1.0 Airflow: Will it work? FAB will read "airflow" one and it will raise deprecation warning, and no way to remove the deprecation warning. Changing the entry to "fab" will not really work (?) because FAB will continue reading "airflow" one. I guess that can be solved by adding FAB >= 2.1.0 in Airfflow 3.1.0

  • Even more interesting is 2.1.0 FAB and 3.0.1 Airflow. FAB will read "fab" entry but airflow wil not have config change inforamtion - so it will only work when you move configuraiton to "fab' but you have no warning about that - as airflow is not aware about that config change. Effect - you might silently loose configuration set in "airflow" config. Again , setting Airlfow >3.1.0 for FAB 2.1.0 makes sense, but then we should make that change just before we relase 3.1.0, because otherwise nobody will be able to use any 2.1.0 FAB after we release it. -until we release Airflow 3.1.0.

In shortl - we have two options:

a) I think we should only make that change just before we release 3.1.0 and bump both Airlfow 3.1.0 and Fab 2.1.0 and make them both depend on the new version.

b) or maybe we shoudl also figure out handling of config change in provider separately so that 2.1.0 FAB will handle config change as well and work also with Airlfow 3.0.*

I think option b) is better if we can pull it off, because then FAB 2.1. will be able to work with Airflow 3.0* and we will be able to relase security fixes independently for it. But then some compatibilty code needs to be implemented in FAB even as simple as manually checking if "airflow" is set and falling back to it - no matter what is implemented in Airflow.

Actually probably best option would be to add possiblity of providers contributing dynamically config changes to Airflow, but that might be a bit too much as we would have to also require some Airflow providers's manager changes and min version set for Airflow in such providers - so likely "hand-written" check for config in FAB provider is a better option:

if "airflow" config exists -> read from there and raise deprecation, else read from "fab". That's probably simples solutin.

@pierrejeambrun
Copy link
Member Author

pierrejeambrun commented May 13, 2025

Even more interesting is 2.1.0 FAB and 3.0.1 Airflow.

In this particular case only fab provider use the config. So basically old core 3.0.1 will not be using it and the new provider will read the config from the provider.

This piece would be problematic if core was using the config option. (or maybe you'r referring to users calling that config option manually ?)

@pierrejeambrun
Copy link
Member Author

pierrejeambrun commented May 13, 2025

I went with #50208 (comment), might be easier for now. So we don't have to wait for anything, change versions before release, or write compatibility code.

@pierrejeambrun pierrejeambrun force-pushed the move-expose_hostname-webserver-config branch from fc84269 to 2fc41a9 Compare May 13, 2025 14:43
@pierrejeambrun pierrejeambrun added the backport-to-v3-0-test Mark PR with this label to backport to v3-0-test branch label May 13, 2025
@pierrejeambrun pierrejeambrun force-pushed the move-expose_hostname-webserver-config branch from 2fc41a9 to 226f7f8 Compare May 13, 2025 15:09
@pierrejeambrun pierrejeambrun force-pushed the move-expose_hostname-webserver-config branch from 226f7f8 to 3d2360a Compare May 13, 2025 16:43
@pierrejeambrun pierrejeambrun merged commit 7c29214 into apache:main May 13, 2025
65 checks passed
@pierrejeambrun pierrejeambrun deleted the move-expose_hostname-webserver-config branch May 13, 2025 17:31
Copy link

Backport failed to create: v3-0-test. View the failure log Run details

Status Branch Result
v3-0-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 7c29214 v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

pierrejeambrun added a commit to astronomer/airflow that referenced this pull request May 14, 2025
@pierrejeambrun
Copy link
Member Author

Manual cherry pick PR #50605

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants