ENH: Add tagging, check, and improved docs

Author: kamil-certat

CHANGELOG.md

@@ -45,6 +45,7 @@
   - Allow saving messages in bulks instead of refreshing the feed immediately (PR#2509 by Kamil Mankowski).
   - Add `attribute_mapping` parameter to allow selecting a subset of event attributes as well as additional attribute parameters (PR#2509 by Kamil Mankowski).
   - Add `event_separator` parameter to allow keeping IntelMQ events in separated MISP Events based on a given field (PR#2509 by Kamil Mankowski).
+  - Add `tagging` parameter to allow adding tags to MISP events (PR#2509 by Kamil Mankowski).
   - Add `additional_info` parameter to extend the default description of MISP Events (PR#2509 by Kamil Mankowski).
 - `intelmq.bots.outputs.smtp_batch.output`: Documentation on multiple recipients added (PR#2501 by Edvard Rejthar).
 

docs/user/bots.md

@@ -4585,6 +4585,12 @@ Create a directory layout in the MISP Feed format.
 The PyMISP library >= 2.4.119.1 is required, see
 [REQUIREMENTS.txt](https://github.com/certtools/intelmq/blob/master/intelmq/bots/outputs/misp/REQUIREMENTS.txt).
 
+Note: please test the produced feed before using in production. This bot allows you to do an
+extensive customisation of the MISP feed, including creating multiple events and tags, but it can
+be tricky to configure properly. Misconfiguration can prevent bot from starting or have bad
+consequences for your MISP Instance (e.g. spaming with events). Use `intelmqctl check` command
+to validate your configuration against common mistakes.
+
 **Module:** `intelmq.bots.outputs.misp.output_feed`
 
 **Parameters:**
@@ -4614,7 +4620,7 @@ hour", string.
 (optional, int) If set to a non-0 value, the bot won't refresh the MISP feed immediately, but will cache
 incoming messages until the given number of them. Use it if your bot proceeds a high number of messages
 and constant saving to the disk is a problem. Reloading or restarting bot as well as generating
-a new MISP event based on `interval_event` triggers saving regardless of the cache size.
+a new MISP event based on `interval_event` triggers regenerating MISP feed regardless of the cache size.
 
 **`attribute_mapping`**
 
@@ -4625,6 +4631,10 @@ dictionary represents additional parameters PyMISP can take when creating an att
 names of other IntelMQ fields (then the value of such field will be used), or static values. If not needed,
 leave empty dict.
 
+For available attribute parameters, refer to the
+[PyMISP documentation](https://pymisp.readthedocs.io/en/latest/_modules/pymisp/mispevent.html#MISPObjectAttribute)
+for the `MISPObjectAttribute`.
+
 For example:
 
 ```yaml
@@ -4668,6 +4678,40 @@ attribute_mapping:
   source.ip:
 ```
 
+**`tagging`
+
+(optional, dict): Allows setting MISP tags to MISP events. The structure is a *dict of list of dicts*.
+The keys refers to which MISP events you want to tag. If you want to tag all of them, use `__all__`.
+If you use `event_separator` and want to add additional tags to some events, use the expected values
+of the separation field. The *list of dicts* defines MISP tags as parameters to create `MISPTag`
+objects from. Each dictonary has to have at least `name`. For all available parameters refer to the
+[PyMISP documentation](https://pymisp.readthedocs.io/en/latest/_modules/pymisp/abstract.html#MISPTag)
+for `MISPTag`.
+
+Note: setting `name` is enough for MISP to match a correct tag from the global collection. You may
+see it lacking the colour in the MISP Feed view, but it will be retriven after importing to your
+instance.
+
+Example 1 - set two tags for every MISP event:
+
+```yaml
+tagging:
+  __all__:
+    - name: tlp:red
+    - name: source:intelmq
+```
+
+Example 2 - create separated events based on `malware.name` and set additional family tag:
+
+```yaml
+event_separator: malware.name
+tagging:
+  __all__:
+    - name: tlp:red
+  njrat:
+    - name: njrat
+```
+
 **Usage in MISP**
 
 Configure the destination directory of this feed as feed in MISP, either as local location, or served via a web server.