Support x-arf

[bernhardreiter]

Intelmq should support x-arf from http://www.x-arf.org
This reading emails and sending emails.

Sending will (most likely) be added to Intevation/intelmq-mailgen#2
first, see progress there.

[aaronkaplan]

okay, we'll leave this open and we can close this ticket as soon as Intevation/intelmq-mailgen#2 gets pulled in.

[bernhardreiter]

Intevation/intelmq-mailgen#2 is only about sending (emails).
We also need the receiving end.
It would be good to have a general mapping between intelMQ objects
and xarf schemas. It should be practical in a python module so it can be used
for reading and writing.

[bernhardreiter]

There are four schemas at http://www.x-arf.org/schemata.html and a few more in https://github.com/abusix/xarf-schemata

[aaronkaplan]

On Tue, May 24, 2016 at 01:05:17AM -0700, bernhardreiter wrote:

Intevation/intelmq-mailgen#2 is only about sending (emails).
We also need the receiving end.
It would be good to have a general mapping between intelMQ objects
and xarf schemas. It should be practical in a python module so it can be used
for reading and writing.

Agreed. This reminds me of our "transformer bot" discussion on the
intelmq-dev list:
https://lists.cert.at/pipermail/intelmq-dev/2016-April/000044.html

[bernhardreiter]

The mapping will be started in Intevation/intelmq-mailgen#2 first, though the goal is to have a mapping in both directions and this would be a standard library that should IMHO go into intelmq itself.

[sebix]

The mapping will be started in Intevation/intelmq-mailgen#2 first, though the goal is to have a mapping in both directions and this would be a standard library that should IMHO go into intelmq itself.

yep, maybe some members on the IHAP list can also comment/review your proposal.

[bernhardreiter]

Just updateded Intevation/intelmq-mailgen#2 , summary:
We proposed a mapping to shadowserver-botnet-drone and feedback suggests that we'll create an updated schema based on abuse_bot-infection_0.1.0.json that includes all values that we consider valuable for the recipient.

@dmth My suggestion is: As long as our new schema version is not "officially" in the x-arf schema, we can put it elsewhere, github or even intel.org would be possible places. It is only until it is merged.

[dmth]

This unstable schema contains more of IntelMQs fields:
https://github.com/Intevation/xarf-schemata/blob/master/abuse_bot-infection_0.2.0_unstable.json

@aaronkaplan could you have a look at the proposed scheme?