-
Notifications
You must be signed in to change notification settings - Fork 70
Description
Describe the bug
We hit an edge case in our production environment where NonBlockingBufferReader#readLines
enters an infinite loop when the input file has lines with different lengths.
To Reproduce
Steps to reproduce the behaviour:
- Generate an input file with two lines:
- First line's length shorter than the NonBlockingBufferReader's buffer
initialCapacity
. - Second line's length
>= initialCapacity
.
- First line's length shorter than the NonBlockingBufferReader's buffer
- parameter
minRecords
passed toNonBlockingBufferReader#readLines
should be> 1
.
Expected behavior
NonBlockingBufferReader should read all lines of the input file.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
In master...lucarosellini:kafka-connect-file-pulse:different-lengths-issue I've added a new test case to reproduce the issue.
To do this, I had to perform some minor refactoring of the test code. These changes are not intended to be merged into master; they are solely for reproducing the issue.
The problem lies in the while loop in readLines()
. If the first line is shorter than the buffer capacity, it is read and added to the records
collection.
The while
loop continues reading input bytes until bufferOffset
equals buffer.length
. At that point, nread
is always 0
because the len
parameter passed to reader.read()
is 0
.
Since records
is not empty (we’ve already read one full line), the if
condition is never true due to the records.isEmpty()
check and the buffer is never expanded.
As a result, the loop becomes infinite.