Merge pull request #2499 from certtools/filter-docs

Filter expert: Docs fix, treat false as false for filter_regex and add logging

Author: sebix

CHANGELOG.md

@@ -34,6 +34,8 @@
 #### Experts
 - `intelmq.bots.experts.sieve.expert`:
   - For `:contains`, `=~` and `!~`, convert the value to string before matching avoiding an exception. If the value is a dict, convert the value to JSON (PR#2500 by Sebastian Wagner).
+- `intelmq.bots.experts.filter.expert`:
+  - Treat value `false` for parameter `filter_regex` as false (PR#2499 by Sebastian Wagner).
 
 #### Outputs
 - `intelmq.bots.outputs.misp.output_feed`: Handle failures if saved current event wasn't saved or is incorrect (PR by Kamil Mankowski).

docs/user/bots.md

@@ -2730,25 +2730,23 @@ A simple filter for messages (drop or pass) based on a exact string comparison o
 
 **`filter_key`**
 
-() - key from data format
+(required, string) - key from data format
 
 **`filter_value`**
 
-() - value for the key
+(required, string) - value for the key
 
 **`filter_action`**
 
-() - action when a message match to the criteria
+(required, string) - action when a message match to the criteria
 (possible actions: keep/drop)
 
 **`filter_regex`**
 
-() - attribute determines if the `filter_value` shall be treated as regular expression or not.
+(optional, boolean) - attribute determines if the `filter_value` shall be treated as regular expression or not.
 
-If this attribute is not empty (can be `true`, `yes` or whatever), the bot uses python's `` `re.search ``
-<<https://docs.python.org/3/library/re.html#re.search>>`_ function to evaluate the filter with regular expressions. If
-this attribute is empty or evaluates to false, an exact string comparison is performed. A check on string *
-inequality* can be achieved with the usage of *Paths* described below.
+If this attribute is not empty (can be `true`, `yes` or whatever), the bot uses python's [`re.search`](https://docs.python.org/3/library/re.html#re.search) function to evaluate the filter with regular expressions. If
+this attribute is empty or evaluates to false, an exact string comparison is performed. A check on string *inequality* can be achieved with the usage of *Paths* described below.
 
 *Parameters for time based filtering*
 

intelmq/bots/experts/filter/expert.py

@@ -64,7 +64,7 @@ def init(self):
             self.filter = False
 
         self.regex = False
-        if self.filter_regex is not None:
+        if self.filter_regex:
             self.regex = re.compile(self.filter_value)
 
         self.time_filter = self.not_after is not None or self.not_before is not None
@@ -148,10 +148,12 @@ def doFilter(self, event, key, condition):
             return self.equalsFilter(event, key, condition)
 
     def equalsFilter(self, event, key, value):
+        self.logger.debug('Equality check: %r (event value) == %r (filter value).', event.get(key), value)
         return (key in event and
                 event.get(key) == value)
 
     def regexSearchFilter(self, event, key):
+        self.logger.debug('Regex filter: Matching %r against %r.', str(event.get(key)), self.filter_value)
         if key in event:
             return self.regex.search(str(event.get(key)))
         else: