Skip to content

wip: ctl: fix logging level filtering #2607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from
Draft

wip: ctl: fix logging level filtering #2607

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Please refer to the [NEWS](NEWS.md) for a list of changes which have an affect o
### Tests

### Tools
- `intelmq.bin.intelmqctl`: Fix bot log level filtering (PR#2607 by Sebastian Wagner, fixes #2596).

### Contrib

Expand Down
4 changes: 3 additions & 1 deletion intelmq/bin/intelmqctl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -794,6 +794,7 @@ def read_bot_log(self, bot_id, log_level, number_of_lines):
message_count = 0

for line in utils.reverse_readline(bot_log_path):
print(line)
if self._parameters.logging_handler == 'file':
log_message = utils.parse_logline(line)
if self._parameters.logging_handler == 'syslog':
Expand All @@ -805,7 +806,8 @@ def read_bot_log(self, bot_id, log_level, number_of_lines):
continue
if log_message['bot_id'] != bot_id:
continue
if LogLevel[log_message['log_level']].value > LogLevel[log_level].value:
print(log_message['log_level'], LogLevel[log_level])
if LogLevel[log_message['log_level']].value < LogLevel[log_level].value:
continue

if message_overflow:
Expand Down
36 changes: 36 additions & 0 deletions intelmq/tests/assets/test-bot.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
2025-04-23 23:17:52,212 - test-bot - INFO - ASNLookupExpertBot initialized with id asn-expert and intelmq 3.4.1.alpha1 and python 3.13.2 (main, Feb 05 2025, 09:57:44) [GCC] as process 257689. Standalone mode: True.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Library path: '/home/sebastianw/dev/intelmq/intelmq/lib/bot.py'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Loading runtime configuration from '/etc/intelmq/runtime.yaml'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'autoupdate_cached_database' loaded with value False.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'destination_pipeline_broker' loaded with value 'redis'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'process_manager' loaded with value 'intelmq'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'source_pipeline_broker' loaded with value 'redis'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'ssl_ca_certificate' loaded with value None.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'statistics_database' loaded with value 3.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'statistics_host' loaded with value '127.0.0.1'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'statistics_password' loaded with value 'HIDDEN'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Defaults configuration: parameter 'statistics_port' loaded with value 6379.
2025-04-23 23:17:52,212 - test-bot - DEBUG - System configuration: parameter 'module' loaded with value 'intelmq.bots.experts.asn_lookup.expert'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Runtime configuration: parameter 'database' loaded with value '/etc/resolv.conf'.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Runtime configuration: parameter 'autoupdate_cached_database' loaded with value True.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Environment configuration: parameter 'paths_no_opt' loaded with value 1.
2025-04-23 23:17:52,212 - test-bot - INFO - Bot is starting.
2025-04-23 23:17:52,212 - test-bot - DEBUG - Loading Harmonization configuration from '/etc/intelmq/harmonization.conf'.
2025-04-23 23:17:52,239 - test-bot - INFO - Loading source pipeline and queue 'asn-expert-queue'.
2025-04-23 23:17:52,240 - test-bot - INFO - Connected to source queue.
2025-04-23 23:17:52,240 - test-bot - INFO - No destination queues to load.
2025-04-23 23:17:52,240 - test-bot - ERROR - Bot initialization failed.
Traceback (most recent call last):
File "/home/sebastianw/dev/intelmq/intelmq/lib/bot.py", line 241, in __init__
self.init()
~~~~~~~~~^^
File "/home/sebastianw/dev/intelmq/intelmq/bots/experts/asn_lookup/expert.py", line 37, in init
self._database = pyasn.pyasn(self.database)
~~~~~~~~~~~^^^^^^^^^^^^^^^
File "/usr/lib64/python3.13/site-packages/pyasn/__init__.py", line 71, in __init__
self._records = self.radix.load_ipasndb(ipasn_file, "")
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
RuntimeError: Error while parsing/adding IPASN database (record: 1)!
2025-04-23 23:17:52,241 - test-bot - DEBUG - Disconnected from source pipeline.
2025-04-23 23:17:52,241 - test-bot - INFO - Bot stopped.
2025-04-23 23:17:52,241 - test-bot - INFO - Bot stopped.
5 changes: 5 additions & 0 deletions intelmq/tests/bin/test_intelmqctl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,11 @@ def test_check_imports_real_bot_module(self):

import_mock.assert_called_once_with("mocked-module")

def test_intelmqctl_log(self):
path = '../' * (self.intelmqctl._parameters.logging_path.count('/')-1) + os.path.dirname(__file__) + '/../assets/test-bot'
assert self.intelmqctl.read_bot_log(path, 'ERROR', 10) == (0, [['ERROR', 'Bot initialization failed.']])
assert self.intelmqctl.read_bot_log(path, 'INFO', 10) == (0, [])


if __name__ == '__main__': # pragma: nocover
unittest.main()
Loading