Disabling auto-refresh from UI has gone missing

[amit-mittal]

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

After upgrading from Airflow v2 to v3, I am not able to see (👀) a button to disable "auto-refresh" of UI. I looked multiple times making sure I am not missing it, but I am for sure not able to see it.

What you think should happen instead?

• Disable auto-refresh from UI
• As an admin, ideally also disable auto-refresh for everyone.

How to reproduce

Upgrade to Airflow v3 and I don't see a toggle like in the image anywhere

Operating System

Official Airflow docker image

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

Why?

[potiuk]

I think - with reactive approach, it's been a deliberate decision to get rid of the old "no-refresh" behaviour. Is there any particular reason why you want to bring it back? What's the reasoning and need ?

[amit-mittal]

I think since upgrading to Airflow v3, we are seeing those continuous polls have made UI less responsive and putting more load on the backend DB. From UX perspective, I would love to keep auto-refresh enabled for our users though.

We are seeing Task Instance page taking > 3 minutes to load. We cleaned up the DB history to keep only 2 years worth of data, but looks like we have to do even more aggressively to make things better. We can't go less than a year because of compliance needs.

[potiuk]

I think if you are seeing such long loads, there is something seriously wrong with your setup - for example you gave too little resources to your db or api server. I suggest you look at what is taking so much time - all queries should be ussing indexes and pagination to make things faster. I f you can track it down _ by i section g what is going wrong - which api calls are taking long and looking at corresponding queries in the dn and look at your resource usage you might find the culprit.

[amit-mittal]

I do agree with you that it shouldn't take that long and there is definitely something wrong with our setup. In pre-prod environments which we have currently upgrade to v3, we allocate less resources, but still it shouldn't have been this slow or get worse after the upgrade (generally performance should improve). Regarding the index use for DB statements, I linked you to the GitHub issue in my previous message where we clearly are not using the index as part of scheduler loop.

I enabled otel tracing, but I couldn't find any traces for the requests from WebServer (I do see traces for Scheduler). Is there a way to get more details about this? Sorry, I couldn't understand what you meant by "i section g what is going wrong - which api calls are taking long and looking at corresponding queries in the dn".

Some details about our setup:

• We are running MySQL 8.x with 16 GB memory and 10 GB InnoDB buffer pool size. Our CPU usage used to be at 10%, but now it hovers at 50%.
• The instance handles around 200 DAGs, not too complicated. In database, we have last 2 years history.
• We have set page limit in Airflow as 100
• AirflowWebServer has 3 GB memory and can go max up to 8 GB
• AirflowScheduler has 4 GB memory and can go max up to 10 GB. We use LocalExecutor. Currently, we have 2 replicas.
• AirflowDAGProcessor has 2 GB memory and can go max up to 5 GB.
• All the DAGs are stored on NFS

Production instance and DBs running Airflow v2.10.x have more resources, but currently the instance running on Airflow v3.0.2 has the above resources. If you have a recommendation (or need more details), that will be of huge help. We can increase the allocated resources and tweak the Airflow configurations.

[potiuk]

Production instance and DBs running Airflow v2.10.x have more resources, but currently the instance running on Airflow v3.0.2 has the above resources. If you have a recommendation (or need more details), that will be of huge help. We can increase the allocated resources and tweak the Airflow configurations.

Unfortunately - I have no time to debug your setup and configuration - you have to remember that here we help people in our free time.

Not sure which Github issue you are talking about - I look at about few hundred issues a day - and it's not very likely I will be helping with that issue. But if refreshing a page takes 3 minutes then you should definitely explain that in the issue including all sql queries and index information you found and explain that this is what happens, because that obviously means that something is wrong. But definitely "disabling auto refresh" is not a solution to the problem.

I convert it to a discussion now and if you have the other issue opened with all the details, then I think you should continue conversation there.

[amit-mittal]

That's fair! #53154 is the GitHub issue I am talking about.

From Airflow v2 to v3, the resource usages have increased, but I don't see any documentation with any new recommendations. Also, I would have expected as the default behavior is being changed, it's reasonable to add those changes under a feature flag and toggle the default (before removing the flag), rather than changing the default behavior without any configuration. I believe increasing auto_refresh_interval is an alternative for now and we won't be removing that setting anytime soon?

But if refreshing a page takes 3 minutes then you should definitely explain that in the issue including all sql queries and index information you found and explain that this is what happens, because that obviously means that something is wrong.

I am willing to share the SQL queries, but is there a way to enable those end-to-end traces or log statements on Airflow that I can then share? At least one query which is taking a long time is below. These queries are so long, that we have to increase the limit to 5000 characters in performance_schema to even see the full query.

SELECT task_instance.rendered_map_index, task_instance.task_display_name, task_instance.id, task_instance.task_id, task_instance.dag_id, task_instance.run_id, task_instance.map_index, task_instance.start_date, task_instance.end_date, task_instance.duration, task_instance.state, task_instance.try_number, task_instance.max_tries, task_instance.hostname, task_instance.unixname, task_instance.pool, task_instance.pool_slots, task_instance.queue, task_instance.priority_weight, task_instance.operator, task_instance.custom_operator_name, task_instance.queued_dttm, task_instance.scheduled_dttm, task_instance.queued_by_job_id, task_instance.last_heartbeat_at, task_instance.pid, task_instance.executor, task_instance.executor_config, task_instance.updated_at, task_instance.context_carrier, task_instance.span_status, task_instance.external_executor_id, task_instance.trigger_id, task_instance.trigger_timeout, task_instance.next_method, task_instance.next_kwargs, task_instance.dag_version_id, dag_version_1.id AS id_1, dag_version_1.version_number, dag_version_1.dag_id AS dag_id_1, dag_version_1.bundle_name, dag_version_1.bundle_version, dag_version_1.created_at, dag_version_1.last_updated, dag_run_1.state AS state_1, dag_run_1.id AS id_2, dag_run_1.dag_id AS dag_id_2, dag_run_1.queued_at, dag_run_1.logical_date, dag_run_1.start_date AS start_date_1, dag_run_1.end_date AS end_date_1, dag_run_1.run_id AS run_id_1, dag_run_1.creating_job_id, dag_run_1.run_type, dag_run_1.triggered_by, dag_run_1.conf, dag_run_1.data_interval_start, dag_run_1.data_interval_end, dag_run_1.run_after, dag_run_1.last_scheduling_decision, dag_run_1.log_template_id, dag_run_1.updated_at AS updated_at_1, dag_run_1.clear_number, dag_run_1.backfill_id, dag_run_1.bundle_version AS bundle_version_1, dag_run_1.scheduled_by_job_id, dag_run_1.context_carrier AS context_carrier_1, dag_run_1.span_status AS span_status_1, dag_run_1.created_dag_version_id
FROM task_instance INNER JOIN dag_run ON dag_run.dag_id = task_instance.dag_id AND dag_run.run_id = task_instance.run_id LEFT OUTER JOIN dag_version ON dag_version.id = task_instance.dag_version_id LEFT OUTER JOIN dag_version AS dag_version_1 ON dag_version_1.id = task_instance.dag_version_id INNER JOIN dag_run AS dag_run_1 ON dag_run_1.dag_id = task_instance.dag_id AND dag_run_1.run_id = task_instance.run_id
WHERE task_instance.dag_id IN (....) ORDER BY CASE WHEN (task_instance.start_date IS NOT NULL) THEN 0 ELSE 1 END, task_instance.start_date DESC, task_instance.id DESC