two offsets for the same log file in pos fluentd

[vishalk663]

In pos file we seen that two offsets for the same log file

eg:

/var/log/containers/abc-555f465779-mxf28_xyz-4d5c3f57bea0287adc4ddfe7805fd878084de905345147706f2c6fa3617c82cd.log	0000000003221c21	00000000224582ae

/var/log/containers/abc-555f465779-mxf28_xyz-4d5c3f57bea0287adc4ddfe7805fd878084de905345147706f2c6fa3617c82cd.log	0000000000000000	0000000000000000

Because of this two offsets, logs of this pod are not pushed into elasticsearch. but after some time duplicate offsets gone. I think it's gone after 21 hrs. because after 21 hrs we received the logs for this specific pod in elasticsearch.

we are using below fluentd image
fluentd image:- fluent/fluentd-kubernetes-daemonset:v1.13.2-debian-elasticsearch7-1.0

[daipom]

This is a known problem:

• Fluentd in_tail "unreadable" file causes "following tail of <file>" to stop and no logs pushed #3614

under certain conditions, in_tail wrongly stops following the new current file (after rotation)
After restarting Fluentd, in_tail starts to collect the file.
We are currently fixing this bug.
The workaround is to set rotate_wait to 0.

rotate_wait 0

[vishalk663]

Thank you for the response.

we are also getting the same error in Flunetd as mentioned in above GitHub issue.

xxxx.log is unreadable. It is excluded and would be examined next time.

How it is automatically resolved after 21 hrs. and we start to receive the logs for this specific pod in elasticsearch

[daipom]

Looks like the file was temporarily inaccessible.
That could be the cause of this bug.

How it is automatically resolved after 21 hrs. and we start to receive the logs for this specific pod in elasticsearch

It is a known behavior as well.
The transfer will stop and fix itself repeatedly.
It occurs when some processes are done at a certain timing and in a certain order.