Filter Anomaly thread

Anomaly thread

Description

It's a filter that receive connection data, stock them in a redis. Every 5 minutes, it fetch redis' data and apply an algorithm on it, to detect anomalies on it.

How it work :

---data---> |anomaly filter| ---add---> (redis) <---query every 5min--- |anomaly thread|

If enough data when anomaly thread query redis:

(redis) <---trim used data--- |anomaly thread| ---write results---> |log file|

Dependencies

• armadillo 9.400 or above
• mlpack 3.0.1 or above

Darwin configuration

Example of darwin configuration for this filter :

{
    "tanomaly_1": {
        "exec_path": "/path/to/darwin/build/darwin_anomaly",
        "config_file":"/path/to/darwin/conf/ftanomaly/ftanomaly.conf",
        "output": "",
        "next_filter": "",
        "nb_thread": 1,
        "log_level": "DEBUG",
        "cache_size": 0
    }
}

Config file

• redis_socket_path : the redis in which the filter will stock the data
• redis_list_name : the key under wich the filter will stock the data in the redis
• log_file_path : the path in which the filter will write it's results

Example :

{
    "redis_socket_path": "/var/sockets/redis/redis.sock",
    "redis_list_name": "anomalyFilter",
    "log_file_path": "/var/log/darwin/anomaly.log"
}

Body

[ 
      ["<net_src_ip>","<net_dst_ip>","<net_dst_port>","<ip_proto>"], 
      [...] 
]

or

{
    "learning_mode": "<mode>",
    "data": [ 
                ["<net_src_ip>","<net_dst_ip>","<net_dst_port>","<ip_proto>"], 
                [...] 
            ]
}

• Where ip_proto need to represent either udp (6), tcp (17), icmp(1). If it's icmp, no need for the field net_dst_port.
• mode can be either on, if you want to start the thread, or off, if you want to stop it.
• learning_mode and data are both optional

Example

If you just want to send some data to the filter :

[     
       ["10.20.255.22", "10.20.255.22", "2831", "6"],
       ["10.1.68.72", "10.1.68.72", "1375", "6"],
       ["10.0.12.101", "10.0.12.101", "2584", "6"],
       ["10.3.36.92", "10.3.36.92", "2802", "6"],
       ["10.12.69.123", "10.12.69.123", "1669", "17"],
       ["172.17.10.12", "172.17.10.12", "1"],
       ["10.1.14.99", "10.1.14.99", "2360", "6"],
       ["10.12.64.234", "10.12.64.234", "1221", "6"],
       ["10.2.74.84", "10.2.74.84", "2902", "17"],
       ["10.6.56.140", "10.6.56.140", "806", "6"],
       ["10.3.25.96", "10.3.25.96", "495", "6"],
       ["10.1.85.210", "10.1.85.210", "1573", "6"],
       ["10.2.5.21", "10.2.5.21", "1178", "6"]
       ["10.12.82.154", "10.12.82.154", "2993", "6"],
       ["10.1.73.83", "10.1.73.83", "80", "6"],
       ["10.3.31.85", "10.3.31.85", "2229", "6"],
       ["10.12.71.91", "10.12.71.91", "763", "6"],
       ["172.19.103.5", "172.19.103.5", "1"], 
       ["10.1.44.71", "10.1.44.71", "2397", "6"],
       ["10.1.31.102", "10.1.31.102", "2696", "6"],
       ["10.1.59.102", "10.1.59.102", "110", "6"],
       ["10.2.89.151", "10.2.89.151", "387", "6"],
       ["10.1.83.183", "10.1.83.183", "1792", "6"],
       ["10.5.22.144", "10.5.22.144", "2046", "6"],
       ["10.0.76.73", "10.0.76.73", "2849", "6"]
]

If you want to start the thread :

{
   "learning_mode": "on"
}

Results

Send no certitude back. Instead when the thread find an anomaly, write logs. Example of log:

{ 
    "time": "2019-07-31UTC12:54:15+0000",
    "anomaly": {
        "ip": "10.5.22.144",
        "udp_nb_host": 16.000000,
        "udp_nb_port": 2.000000,
        "tcp_nb_host": 520.000000,
        "tcp_nb_port": 4.000000,
        "icmp_nb_host": 0.000000,
        "distance": 624.351380
    }
}

Configuration with Rsyslog

Modules necessary

To work alongside this filter, Rsyslog must be used with those compiled modules:

• impcap
• mmdarwin

Impcap

The Impcap module is a network sniffing module, it will recover information and data from packets on the network. Please go to its corresponding wiki to get full documentation on how to configure it.

Mmdarwin

The Mmdarwin module comes after Impcap, and handles the transmission of data to darwin, the configuration of Mmdarwin in the rsyslog configuration file should be:

action(type="mmdarwin" socketpath="/path/to/darwin/sockets/tanomaly_1.sock" fields=["!impcap!net_src_ip", "!impcap!net_dst_ip", "!ipmcap!net_dst_port", "!impcap!IP_proto"] key="TANOMALY" filtercode="0x544D4C59")