Open
Description
If the sieve filtering script has a line like if extra.min_amplification < 5, the sieve won't start but instead throw an error like:
2025-04-13 03:59:06,415 - shadowserver-sieve-expert - ERROR - Bot initialization failed.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 532, in validate_numeric_match
type = SieveExpertBot._harmonization[num_match.key]["type"]
KeyError: 'extra.min_amplification'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 265, in read_sieve_file
sieve = metamodel.model_from_file(filename)
File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 661, in model_from_file
return self.internal_model_from_file(
File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 709, in internal_model_from_file
model = self._parser_blueprint.clone().get_model_from_str(
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 381, in get_model_from_str
model = parse_tree_to_objgraph(
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 903, in parse_tree_to_objgraph
call_obj_processors(m._tx_metamodel, m)
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 759, in call_obj_processors
result = call_obj_processors(metamodel,
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 765, in call_obj_processors
result = call_obj_processors(metamodel,
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 765, in call_obj_processors
result = call_obj_processors(metamodel,
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 759, in call_obj_processors
result = call_obj_processors(metamodel,
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 759, in call_obj_processors
result = call_obj_processors(metamodel,
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 765, in call_obj_processors
result = call_obj_processors(metamodel,
File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 775, in call_obj_processors
return_value_current = metamodel.process(
File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 530, in process
raise e
File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 520, in process
return self._obj_processors.get(_type, lambda x: x)(value)
File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 536, in validate_numeric_match
raise TextXSemanticError(f"Invalid key: {num_match.key}.", **position)
textx.exceptions.TextXSemanticError: /etc/intelmq/shadowserver-sieve-expert.sieve:383:74: Invalid key: extra.min_amplification.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/intelmq/lib/bot.py", line 241, in __init__
self.init()
File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 70, in init
self.model = self.read_sieve_file(self.file, self.metamodel)
File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 268, in read_sieve_file
raise ValueError(
ValueError: Could not parse sieve file '/etc/intelmq/shadowserver-sieve-expert.sieve', error in (383, 74): /etc/intelmq/shadowserver-sieve-expert.sieve:383:74: Invalid key: extra.min_amplification.
The reason is that validate_numeric_match method look for the harmonization field to ensure the right numeric type. However, we do not have types for the extra fields, even if they are properly defined.
As so, I suggest not to statically validate if the extra.* field is a numeric, but instead rely on runtime exceptions for that.