Skip to content

Commit 7d2012d

Browse files
committed
update NEWS.md in preparation for 3.3.0
1 parent 618d2ba commit 7d2012d

File tree

1 file changed

+52
-7
lines changed

1 file changed

+52
-7
lines changed

NEWS.md

Lines changed: 52 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,20 +14,65 @@ Please refer to the change log for a full list of changes.
1414
---------------------------------
1515

1616
### Documentation
17-
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements.
17+
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements. Big thanks to to @gethvi.
18+
We now have a slick, modern mkdocs based documentation. Please do check it out!
1819

19-
### Requirements
2020

21-
### Tools
21+
### Bots
22+
#### Shadowserver dynamic parser / collector
2223

23-
### Data Format
24+
**Note well**: if you use shadowserver feeds, **please read this section carefully**.
2425

25-
### Configuration
26+
Thanks to shadowserver (@elsif2), we have a new dynamic shadowserver reports API integration. What does it do?
27+
It connects to the [Shadowserver API](https://www.shadowserver.org/what-we-do/network-reporting/api-documentation/),
28+
requests a list of all the reports for a specific country and processes the ones that are new.
2629

27-
### Libraries
30+
Motivation for this change:
2831

29-
### Postgres databases
32+
Shadowserver adds new scans on a nearly weekly basis. IntelMQ's release cycle and the need for a stable release could not keep up with this high intensity of shadowserver parser changes.
33+
We therefore (thanks to @eslif2) move the shadowserver reports collector and parser to a new, dynamic system. It can:
34+
35+
- fetch the shadowserver schema from shadowserver (https://interchange.shadowserver.org/intelmq/v1/schema)
36+
- dynamically collect new reports (see also https://docs.intelmq.org/latest/user/bots/?h=shadow#shadowserver-reports-api)
37+
- parse the new reports
38+
39+
**Note well**: if your IntelMQ system runs in an airgapped environment or if it may only reach out to specific IPs/sites, you should read the notes here:
40+
https://docs.intelmq.org/latest/user/bots/#shadowserver.
41+
You will need to download shadowserver-schema.json periodically yourself in this case.
42+
43+
**Note well:**: since dynamic changes are a bit tricky, we defined that there is a schema contract:
44+
45+
> Schema contract
46+
>
47+
> Once set in the schema, the classification.identifier, classification.taxonomy, and classification.type fields will remain static for a specific report.
48+
49+
This makes things deterministic again.
50+
51+
#### Alienvault OTX
52+
53+
Fix of a bug where a certain condition would have always evaluated to False. (PR#2449 by qux-bbb. Thanks)
54+
55+
#### AMQP
56+
Quite a few changes (thanks to Kamil, @gethvi) on AMQP
57+
58+
#### Obsoleted bots
59+
60+
- `intelmq.bots.parsers.netlab_360.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
61+
- `intelmq.bots.parsers.webinspektor.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
62+
- `intelmq.bots.parsers.sucuri.parser`: Removed as the feed is discontinued. (#2442 by Filip Pokorný)
63+
64+
### General changes and bug fixes
65+
66+
Digital Trust Center fixed a bug where the config was loaded twice in intelmqctl which created quite some speedups. Thanks!
67+
This speeds up IntelMQ API calls.
68+
69+
### Data Format
70+
71+
Shadowserver dynamic parser (see above).
72+
73+
### General remarks
3074

75+
The full list of changes can be seen in the CHANGELOG.md file.
3176

3277
3.2.1 Bugfix release (2023-08-23)
3378
---------------------------------

0 commit comments

Comments
 (0)