Skip to content

Fix: airflow.task log level default to INFO #649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 26, 2025
Merged

Fix: airflow.task log level default to INFO #649

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
16c66ad
set INFO as default for airflow.task logging
Maleware Jun 25, 2025
ed7d743
Add documentation
Maleware Jun 25, 2025
8c04906
Add example to docs
Maleware Jun 25, 2025
ef76f35
Using handler task to set log level, update comment
Maleware Jun 25, 2025
bb842e5
Moving handler to comment
Maleware Jun 25, 2025
45bcae7
update Changelog.md
Maleware Jun 25, 2025
cc013e3
fixing typo
Maleware Jun 25, 2025
d2061bb
Better documentation
Maleware Jun 26, 2025
a0378e8
More robust decision makeing
Maleware Jun 26, 2025
94b4a67
Merge branch 'fix/adopt-logging-default' of https://github.com/stacka…
Maleware Jun 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- Use `--console-log-format` (or `CONSOLE_LOG_FORMAT`) to set the format to `plain` (default) or `json`.
- Add support for airflow `2.10.5` ([#625]).
- Add experimental support for airflow `3.0.1` ([#630]).
- "airflow.task" logger defaults to log level 'INFO' instead of 'NOTSET' ([#649]).

### Changed

Expand Down Expand Up @@ -50,6 +51,7 @@
[#630]: https://github.com/stackabletech/airflow-operator/pull/630
[#636]: https://github.com/stackabletech/airflow-operator/pull/636
[#645]: https://github.com/stackabletech/airflow-operator/pull/645
[#649]: https://github.com/stackabletech/airflow-operator/pull/649

## [25.3.0] - 2025-03-21

Expand Down
7 changes: 7 additions & 0 deletions docs/modules/airflow/pages/usage-guide/logging.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@

The logs can be forwarded to a Vector log aggregator by providing a discovery ConfigMap for the aggregator and by enabling the log agent:

NOTE: The `task` handler is responsible for showing the task logs in the UI.
Unfortunately, the log level of the `task` handler cannot be specified.
To avoid that all logs are emitted to the UI, the log level of the `airflow.task` logger is set explicitly to `INFO`.
You can change the log level as shown below.

[source,yaml]
----
spec:
Expand All @@ -26,6 +31,8 @@ spec:
loggers:
"airflow.processor":
level: INFO
"airflow.task":
level: DEBUG
schedulers:
config:
logging:
Expand Down
5 changes: 5 additions & 0 deletions rust/operator-binary/src/product_logging.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,11 @@ for logger_name, logger_config in LOGGING_CONFIG['loggers'].items():
# otherwise DAGs cannot be loaded anymore.
if logger_name != 'airflow.task':
logger_config['propagate'] = True
# The default behavior of airflow is to enforce log level 'INFO' on tasks. (https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#logging-level)
# TODO: Make task handler log level configurable through CRDs with default 'INFO'.
# e.g. LOGGING_CONFIG['handlers']['task']['level'] = {{task_log_level}}
if 'handlers' in logger_config and 'task' in logger_config['handlers']:
logger_config['level'] = logging.INFO

LOGGING_CONFIG.setdefault('formatters', {{}})
LOGGING_CONFIG['formatters']['json'] = {{
Expand Down