Sieve does not support numeric comparation in extra.*

[kamil-certat]

If the sieve filtering script has a line like if extra.min_amplification < 5, the sieve won't start but instead throw an error like:

2025-04-13 03:59:06,415 - shadowserver-sieve-expert - ERROR - Bot initialization failed.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 532, in validate_numeric_match
    type = SieveExpertBot._harmonization[num_match.key]["type"]
KeyError: 'extra.min_amplification'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 265, in read_sieve_file
    sieve = metamodel.model_from_file(filename)
  File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 661, in model_from_file
    return self.internal_model_from_file(
  File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 709, in internal_model_from_file
    model = self._parser_blueprint.clone().get_model_from_str(
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 381, in get_model_from_str
    model = parse_tree_to_objgraph(
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 903, in parse_tree_to_objgraph
    call_obj_processors(m._tx_metamodel, m)
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 759, in call_obj_processors
    result = call_obj_processors(metamodel,
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 765, in call_obj_processors
    result = call_obj_processors(metamodel,
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 765, in call_obj_processors
    result = call_obj_processors(metamodel,
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 759, in call_obj_processors
    result = call_obj_processors(metamodel,
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 759, in call_obj_processors
    result = call_obj_processors(metamodel,
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 765, in call_obj_processors
    result = call_obj_processors(metamodel,
  File "/usr/local/lib/python3.9/dist-packages/textx/model.py", line 775, in call_obj_processors
    return_value_current = metamodel.process(
  File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 530, in process
    raise e
  File "/usr/local/lib/python3.9/dist-packages/textx/metamodel.py", line 520, in process
    return self._obj_processors.get(_type, lambda x: x)(value)
  File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 536, in validate_numeric_match
    raise TextXSemanticError(f"Invalid key: {num_match.key}.", **position)
textx.exceptions.TextXSemanticError: /etc/intelmq/shadowserver-sieve-expert.sieve:383:74: Invalid key: extra.min_amplification.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/intelmq/lib/bot.py", line 241, in __init__
    self.init()
  File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 70, in init
    self.model = self.read_sieve_file(self.file, self.metamodel)
  File "/usr/local/lib/python3.9/dist-packages/intelmq/bots/experts/sieve/expert.py", line 268, in read_sieve_file
    raise ValueError(
ValueError: Could not parse sieve file '/etc/intelmq/shadowserver-sieve-expert.sieve', error in (383, 74): /etc/intelmq/shadowserver-sieve-expert.sieve:383:74: Invalid key: extra.min_amplification.

The reason is that validate_numeric_match method look for the harmonization field to ensure the right numeric type. However, we do not have types for the extra fields, even if they are properly defined.

As so, I suggest not to statically validate if the extra.* field is a numeric, but instead rely on runtime exceptions for that.

[sebix]

As so, I suggest not to statically validate if the extra.* field is a numeric, but instead rely on runtime exceptions for that.

For all fields with type "JSONDict"