Meraki Cloud Syslog to Firehose

[bradkelso32]

Describe the bug

Hello,

I'm trying to use fluentd to get Meraki syslog logs into AWS firehose. I installed this plugin for firehose: https://github.com/awslabs/aws-fluent-plugin-kinesis

Meraki syslog doc: https://documentation.meraki.com/General_Administration/Monitoring_and_Reporting/Syslog_Server_Overview_and_Configuration

I keep getting this error:
2023-03-29 01:39:50 +0000 [error]: #0 invalid input data="<1234>1 1680053990.249297089 something_appliance ip_flow_start src=1.2.3.4 dst=1.2.3.4 protocol=udp sport=1234 dport=1234 translated_src_ip=1.2.3.4 translated_port=1234" error_class=Fluent::TimeParser::TimeParseError error="invalid time format: value = 1 1680053990.249297089 something_appliance, error_class = ArgumentError, error = string doesn't match"

To Reproduce

configure meraki syslog, install firehose plugin, configure conf file

Expected behavior

No errors

Your Environment

- Fluentd version: 1.15.3
- TD Agent version: 4.4.2
- Operating system: Ubuntu 22.04.2 LTS
- Kernel version: 5.15.0-1031-aws

Your Configuration

<source>
  @type syslog
  port 5140
  bind 0.0.0.0
  tag meraki
</source>

<match syslog.**>
  @type kinesis_firehose
  region region
  delivery_stream_name name

  <assume_role_credentials>
    duration_seconds 3600
    role_arn role
    role_session_name "#{Socket.gethostname}-something"
  </assume_role_credentials>
</match>

Your Error Log

2023-03-29 01:39:50 +0000 [error]: #0 invalid input data="<1234>1 1680053990.249297089 something_appliance ip_flow_start src=1.2.3.4 dst=1.2.3.4 protocol=udp sport=1234 dport=1234 translated_src_ip=1.2.3.4 translated_port=1234" error_class=Fluent::TimeParser::TimeParseError error="invalid time format: value = 1 1680053990.249297089 something_appliance, error_class = ArgumentError, error = string doesn't match"

Additional context

Is there a way to get the raw logs from Meraki without parsing or formatting?

[daipom]

Thanks for reporting, but this is not a bug. So I transferred this issue to discussions.

2023-03-29 01:39:50 +0000 [error]: #0 invalid input data="<1234>1 1680053990.249297089 something_appliance ip_flow_start src=1.2.3.4 dst=1.2.3.4 protocol=udp sport=1234 dport=1234 translated_src_ip=1.2.3.4 translated_port=1234" error_class=Fluent::TimeParser::TimeParseError error="invalid time format: value = 1 1680053990.249297089 something_appliance, error_class = ArgumentError, error = string doesn't match"

This message format looks like rfc5424.

• https://www.rfc-editor.org/rfc/rfc5424

So you should set message_format to rfc5424.

However, perhaps that's not enough to parse this.
The header <1234> looks like PRI of syslog, but this does not follow the format of rfc5424.

• https://www.rfc-editor.org/rfc/rfc5424#section-6.2.1

This number must be at most 3 digits.

[bradkelso32]

Sorry, the header should be 3 numbers ex. <123>

Here is the new conf file:

<source>
  @type syslog
  port 5140
  bind 0.0.0.0
  <parse>
    message_format rfc5424
    rfc5424_time_format %s.%L
  </parse>
  tag meraki
</source>

<match meraki.**>
  @type kinesis_firehose
  region region
  delivery_stream_name name

  <instance_profile_credentials>
  </instance_profile_credentials>
</match>

It generates a new error:
2023-03-29 04:14:56 +0000 [warn]: #0 failed to parse message data="<134>1 1680063296.242261792 something_appliance firewall src=1.2.3.4 dst=1.2.3.4 mac=00:00:00:00:00:00 protocol=tcp sport=1234 dport=1234 pattern: allow (src 1.2.3.424)"

"failed to parse message data" seems to be a generic error, any ideas?

[daipom]

"failed to parse message data" seems to be a generic error, any ideas?

This is a warning from in_syslog.
This means that the incoming message does not follow the correct format.

On in_syslog implementation, the incoming message must match the following regular expression

(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])+))(?: (?<message>.+))?\z

fluentd/lib/fluent/plugin/parser_syslog.rb

Line 47 in e89092c

RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])+))(?: (?<message>.+))?\z/m

Hmm, in addition to PRI, other headers look like not following rfc5424...

As I was concerned here, the incoming message does not yet follow rfc5424.

• https://www.rfc-editor.org/rfc/rfc5424#section-6

PRI timestamp hostname APP-NAME PROCID MSGID STRUCTURED-DATA MSG

Example

• https://www.rfc-editor.org/rfc/rfc5424#section-6.5

<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8

The current message

<134>1 1680063296.242261792 something_appliance firewall src=1.2.3.4 dst=1.2.3.4 mac=00:00:00:00:00:00 protocol=tcp sport=1234 dport=1234 pattern: allow (src 1.2.3.424)

looks like

PRI timestamp hostname APP-NAME MSG

For example, the following message can be parsed.
(none for PROCID and MSGID, and - for STRUCTURED-DATA)

<134>1 1680063296.242261792 something_appliance firewall none none - src=1.2.3.4 dst=1.2.3.4 mac=00:00:00:00:00:00 protocol=tcp sport=1234 dport=1234 pattern: allow (src 1.2.3.424)

[bradkelso32]

Would there be a way to not parse or format at all with fluentd? Just capture the raw logs and send it to aws firehose.

[daipom]

https://docs.fluentd.org/input/syslog#our-system-sends-rfc3164-rfc5424-message-but-parse-failure-happens

You can try another parser, such as regexp.

• https://docs.fluentd.org/parser/regexp#example

<source>
  @type syslog
  port 5140
  bind 0.0.0.0
  <parse>
    @type regexp
    ...
  </parse>
  tag meraki
</source>

Or you can try in_udp.

• https://docs.fluentd.org/input/udp

[daipom]

If you don't need parsing at all, perhaps parser_none can be used too.

• https://docs.fluentd.org/parser/none