You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: NEWS.md
+52-7Lines changed: 52 additions & 7 deletions
Original file line number
Diff line number
Diff line change
@@ -14,20 +14,65 @@ Please refer to the change log for a full list of changes.
14
14
---------------------------------
15
15
16
16
### Documentation
17
-
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements.
17
+
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements. Big thanks to to @gethvi.
18
+
We now have a slick, modern mkdocs based documentation. Please do check it out!
18
19
19
-
### Requirements
20
20
21
-
### Tools
21
+
### Bots
22
+
#### Shadowserver dynamic parser / collector
22
23
23
-
### Data Format
24
+
**Note well**: if you use shadowserver feeds, **please read this section carefully**.
24
25
25
-
### Configuration
26
+
Thanks to shadowserver (@elsif2), we have a new dynamic shadowserver reports API integration. What does it do?
27
+
It connects to the [Shadowserver API](https://www.shadowserver.org/what-we-do/network-reporting/api-documentation/),
28
+
requests a list of all the reports for a specific country and processes the ones that are new.
26
29
27
-
### Libraries
30
+
Motivation for this change:
28
31
29
-
### Postgres databases
32
+
Shadowserver adds new scans on a nearly weekly basis. IntelMQ's release cycle and the need for a stable release could not keep up with this high intensity of shadowserver parser changes.
33
+
We therefore (thanks to @eslif2) move the shadowserver reports collector and parser to a new, dynamic system. It can:
34
+
35
+
- fetch the shadowserver schema from shadowserver (https://interchange.shadowserver.org/intelmq/v1/schema)
36
+
- dynamically collect new reports (see also https://docs.intelmq.org/latest/user/bots/?h=shadow#shadowserver-reports-api)
37
+
- parse the new reports
38
+
39
+
**Note well**: if your IntelMQ system runs in an airgapped environment or if it may only reach out to specific IPs/sites, you should read the notes here:
You will need to download shadowserver-schema.json periodically yourself in this case.
42
+
43
+
**Note well:**: since dynamic changes are a bit tricky, we defined that there is a schema contract:
44
+
45
+
> Schema contract
46
+
>
47
+
> Once set in the schema, the classification.identifier, classification.taxonomy, and classification.type fields will remain static for a specific report.
48
+
49
+
This makes things deterministic again.
50
+
51
+
#### Alienvault OTX
52
+
53
+
Fix of a bug where a certain condition would have always evaluated to False. (PR#2449 by qux-bbb. Thanks)
54
+
55
+
#### AMQP
56
+
Quite a few changes (thanks to Kamil, @gethvi) on AMQP
57
+
58
+
#### Obsoleted bots
59
+
60
+
-`intelmq.bots.parsers.netlab_360.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
61
+
-`intelmq.bots.parsers.webinspektor.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
62
+
-`intelmq.bots.parsers.sucuri.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
63
+
64
+
### General changes and bug fixes
65
+
66
+
Digital Trust Center fixed a bug where the config was loaded twice in intelmqctl which created quite some speedups. Thanks!
67
+
This speeds up IntelMQ API calls.
68
+
69
+
### Data Format
70
+
71
+
Shadowserver dynamic parser (see above).
72
+
73
+
### General remarks
30
74
75
+
The full list of changes can be seen in the CHANGELOG.md file.
0 commit comments