bug: ClusterMultilineParser not being registered by Fluent Operator

[khantnaingset-kns]

Context

When deploying a custom multiline parser configuration, the parser is not being recognized or applied by the FluentBit instances managed by the operator. The Fluent Operator is not properly registering custom ClusterMultilineParser resources, causing container to failed to start.

Expected Behavior

The ClusterMultilineParser should be:

• Successfully registered by the Fluent Operator
• Available for use in ClusterInput configurations via the multilineParser field

Actual Behavior

• The ClusterMultilineParser resource is created but not registered/recognized
• FluentBit instances do not receive the custom parser configuration which cause the container to failed to start

** Code **

---
# Custom Multiline Parser for JSON logs
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterMultilineParser
metadata:
  name: spark-json-multiline
  labels:
    app.kubernetes.io/name: fluent-bit
    app.kubernetes.io/component: fluent-bit-config
spec:
  type: regex
  flushTimeout: 2000
  rules:
    - start: '/^[IWEF]\d{4} \d{2}:\d{2}:\d{2}\.\d+/'
      regex: '^[IWEF]\d{4} \d{2}:\d{2}:\d{2}\.\d+'
      next: cont_state
    - start: '/^(?![IWEF]\d{4})/'
      regex: '^(?![IWEF]\d{4})'
      next: cont_state

---
# Updated ClusterInput with custom multiline parser
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterInput
metadata:
  name: fluent-bit-spark-operator
  labels:
    app.kubernetes.io/name: fluent-bit
    app.kubernetes.io/component: fluent-bit-config
spec:
  tail:
    tag: kube.*
    path: /var/log/containers/*-spark-operator-*.log
    parser: cri
    multilineParser: spark-json-multiline
    refreshIntervalSeconds: 10
    bufferChunkSize: 32k
    bufferMaxSize: 64k
    memBufLimit: 5MB
    skipLongLines: false
    readFromHead: false
    db: /fluent-bit/tail/spark-operator-pos.db
    dbSync: Normal

---
# ClusterFilter for multiline processing
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterFilter
metadata:
  name: multiline-filter
  labels:
    app.kubernetes.io/name: fluent-bit
    app.kubernetes.io/component: fluent-bit-config
spec:
  match: kube.*
  filters:
  - multiline:
      keyContent: log
      buffer: true
      emitterMemBufLimit: 120
      emitterType: memory
      flushMs: 2000
      parser: go,cri,spark-json-multiline

---
# ClusterFluentBitConfig to tie everything together
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterFluentBitConfig
metadata:
  name: fluent-bit-config
  labels:
    app.kubernetes.io/name: fluent-bit
    app.kubernetes.io/component: fluent-bit-config
spec:
  namespace: kube-system
  service:
    parsersFile: "parsers.conf"
  inputSelector:
    matchLabels:
      app.kubernetes.io/component: fluent-bit-config
  filterSelector:
    matchLabels:
      app.kubernetes.io/component: fluent-bit-config
  multilineParserSelector:
    matchLabels:
      app.kubernetes.io/component: fluent-bit-config
  outputSelector:
    matchLabels:
      app.kubernetes.io/component: fluent-bit-config

Log

 [2025/06/06 05:34:32] [ info] [fluent bit] version=2.2.2, commit=eeea396e88, pid=14                                                             │
│ [2025/06/06 05:34:32] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128                                    │
│ [2025/06/06 05:34:32] [ info] [cmetrics] version=0.6.6                                                                                          │
│ [2025/06/06 05:34:32] [ info] [ctraces ] version=0.4.0                                                                                          │
│ [2025/06/06 05:34:32] [ info] [input:tail:tail.0] initializing                                                                                  │
│ [2025/06/06 05:34:32] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)                                                       │
│ [2025/06/06 05:34:32] [error] [multiline] parser 'spark-json-multiline' not registered                                                          │
│ [2025/06/06 05:34:32] [error] [input:tail:tail.0] could not load multiline parsers                                                              │
│ [2025/06/06 05:34:32] [error] failed initialize input tail.0                                                                                    │
│ [2025/06/06 05:34:32] [error] [engine] input initialization failed                                                                              │
│ [2025/06/06 05:34:32] [error] [lib] backend failed                                                                                              │
│ level=error time=2025-06-06T05:34:33Z msg="Failure during the run time of fluent-bit" error="failed to run fluent-bit: exit status 255"  

How did you install fluent operator?

Fluent Operator Version: 2.9.0
Kubernetes Version: v1.30.11
FluentBit Version: 2.2.2

Additional context

No response

[hsmade]

Seeing the same issue. What I can add is that the fluent-bit.conf that is rendered only has a Parsers_File pointing to parsers.conf, while the multiline parser gets added to parsers_multiline.conf.
This is probably why fluent doesn't know about it.

[hsmade]

From some quick testing, this could be solved by just adding the parsers_multiline.conf to the fluentbit-config. It will throw an error when the file doesn't exist, but it does just continue (and loads other files just fine). It's not the prettiest solution.
An alternative would be that the controller would insert the Parsers_File line if it did find a MultilineParser or a ClusterMultilineParser resource.

[cw-Guo]

i haven't tested it myself, but from my understanding, if you use the latest fluentbit image built by fluent-operator, it should be fine, since we included the multiline parser here:

fluent-operator/conf/parsers.conf

Line 2 in 59ac9ea

@INCLUDE /fluent-bit/config/parsers_multiline.conf