Getting Parse error in Source

[sudhanshud]

Hi All ,

I am getting below error in my fluentd source.
2023-07-07T10:06:30.135630Z ERROR source{component_kind="source" component_id=fluent component_type=fluent component_name=fluent}:connection{peer_addr=XXXX:62907}: vector::internal_events::codecs: Internal log [Failed framing bytes.] has been rate limited 6 times.
2023-07-07T10:06:30.135646Z ERROR source{component_kind="source" component_id=fluent component_type=fluent component_name=fluent}:connection{peer_addr=XXXX:62907}: vector::internal_events::codecs: Failed framing bytes. error=bytes remaining on stream error_type="parser_failed" stage="processing" internal_log_rate_limit=true

Any idea about the issue?

[jszwedko]

Can you share your FluentD configuration and version? A full minimal reproducible example here would be very helpful. You could also consider using the HTTP FluentD output with the HTTP Vector source.

[sudhanshud]

Fluent-Bit Version is 2.1.6
Below is the fluent bit configuration.

[SERVICE]
    Daemon Off
    Flush {{ .Values.flush }}
    Log_Level {{ .Values.logLevel }}
    Parsers_File parsers.conf
    Parsers_File custom_parsers.conf
    HTTP_Server On
    HTTP_Listen 0.0.0.0
    HTTP_Port {{ .Values.metricsPort }}
    Health_Check On
[INPUT]
    Name tail
    Path /var/log/containers/*.log
    Tag kube.*
    Parser cri-parser
    Mem_Buf_Limit 5MB
    Skip_Long_Lines On
[FILTER]
    Name kubernetes
    Match kube.*
    Merge_Log On
    Keep_Log Off
    K8S-Logging.Parser On
    K8S-Logging.Exclude On
[OUTPUT]
    Name  forward
    Match kube.*
    Host  vector-aggregator.niyo.ops
    Port  24226
    Tag UAT-NIYO
[PARSER]
    Name cri-parser
    Format regex
    Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
    Time_Key  timestamp
    Time_Format %Y-%m-%dT%H:%M:%S.%L%z        
[MULTILINE_PARSER]
    Name          multiline-regex-cri
    Type          regex
    Flush_Timeout 1000
    Rule      "start_state"   "/(\D+ \d+ \d+\:\d+\:\d+)(.*)/"  "cont"
    Rule      "cont"          "/^\s+at.*/"                     "cont"

[sudhanshud]

@jszwedko Any idea why this happening?
I did not get any idea.

[jszwedko]

Thanks for sharing! I wasn't able to reproduce this yet. I'd suggest trying the http fluentbit output with the http_server Vector source.