Parsing Nested Json Log

[allahshukur-ahmadzada]

Can fluentd parse nested json log? if yes can anyone share an exmple? like at the fields should be nested, host.name , host.os and so on

[daipom]

The json parser can parse nested JSON log.

• https://docs.fluentd.org/parser/json

For example, we can format the parsed JSON log by using fitler_record_transformer as follows

Example 1

Config

<source>
  @type tcp
  tag test
  <parse>
    @type json
  </parse>
</source>

<filter test.**>
  @type record_transformer
  enable_ruby true
  <record>
    fuga ${record.dig("hoge", "fuga")}
  </record>
</filter>

<match test.**>
  @type stdout
  <format>
    @type csv
    fields fuga, foo
  </format>
</match>

Send nested JSON

$ netcat 0.0.0.0 5170
{"hoge":{"fuga":1},"foo":2}

Result

"1","2"

Example 2

Config

<source>
  @type tcp
  tag test
  <parse>
    @type json
  </parse>
</source>

<filter test.**>
  @type record_transformer
  enable_ruby true
  renew_record true
  <record>
    fuga ${record.dig("hoge", "fuga")}
    foo ${record["foo"]}
  </record>
</filter>

<match test.**>
  @type stdout
</match>

Send nested JSON

$ netcat 0.0.0.0 5170
{"hoge":{"fuga":1},"foo":2}

Result

2023-03-09 01:15:03.453214797 +0900 test: {"fuga":1,"foo":2}