Parse IIS access log

[atomotic]

Given an IIS access log (below an excerpt, anonymized), what is the suggested way to parse this and have a message with all fields? Should I use parse_regex!, can you point me at any example?

#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2024-03-04 11:00:00
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2024-03-04 11:00:00 10.0.0.1 GET /url1 module=5378 443 - 0.0.0.0 Mozilla/5.0+(iPhone;+CPU+iPhone+OS+17_3_1+like+Mac+OS+X)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/17.3.1+Mobile/15E148+Safari/604.1 https://referer1 200 0 0 3569
2024-03-04 11:00:00 10.0.0.1 GET /url2 v=799 443 - 0.0.0.0 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/121.0.0.0+Safari/537.36+OPR/107.0.0.0 https://referer2 200 0 0 106

[alexpaknix]

Hi, I'm use this setting

    type: remap
    inputs:
      - iis_logs
    source: | #                      \/ This flag makes regex multiline.
      . |= parse_regex!(.message, r'(?m)^(?P<timestamp>\d+-\d+-\d+ \d+:\d+:\d+) (?P<sIp>\d+.\d+.\d+.\d+) (?P<method>\w+) (?P<csUri>\S+) (?P<csUriQuery>\S+) (?P<sPort>\d+) (?P<csUserName>\S+) (?P<cIp>\d+.\d+.\d+.\d+) (?P<csUA>\S+) (?P<csR>\S+) (?P<csStatus>\d+) (?P<csSubstatus>\d+) (?P<csWin32Status>\d+) (?P<timeTaken>\d+)')

[vividDuck]

This is the remap I've used and works for an iis log format of:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken X-Forwarded-For

Basically they are space separated, so you could use the parse_csv, parse_csv!(.message, delimiter: " "). But that would not provide a header.

remap_iis_logs:
    inputs:
      - g_iis_logs
    type: remap
    source: |
            . |= parse_regex!(.message, r'(?P<datetime>[^ ]+ [^ ]+) (?P<source_ip>[^ ]+) (?P<method>[^ ]+) (?P<uri_stem>[^ ]+) (?P<uri_query>[^ ]+) (?P<source_port>[^ ]+) (?P<username>[^ ]+) (?P<client_ip>[^ ]+) (?P<version>[^ ]+) (?P<user_agent>[^ ]+) (?P<referer>[^ ]+) (?P<status>[^ ]+) (?P<sub_status>[^ ]+) (?P<win32_status>[^ ]+) (?P<source_bytes>[^ ]+) (?P<client_bytes>[^ ]+) (?P<time_taken>[^ ]+) (?P<x_forwarded_for>[^ \r]+)')
            .datetime = parse_timestamp(.datetime, format: "%F %T") ?? now()