How to process files with only one line and no line breaks?

[pirxthepilot]

I have vector monitoring a directory that an app writes event logs to. Each file contains exactly one event in JSON format, which contains exactly 1 line, but does not have a newline at the end. (When you wc -l this file it will say it has 0 lines)

I found that vector does not process the line if it does not have that newline. When the file gets created, vector does see it and emits "Found new file to watch" but it does not go further. It will parse the line if I echo >> logfile.json (essentially adding that newline).

I also tried simulating this logging process with python and was able to replicate the issue. If I add a newline on file write, vector does not have a problem parsing the line.

Is there a way to solve this? Thanks in advance!

[bruceg]

The Vector file source is designed for handling files that may be being written to while we are reading. This means it needs some kind of marker to tell when a record has been completely written. By default, that marker is newline. Without the marker, Vector can't know if the line is actually complete or that the writer is still stuck waiting to write something else to the file.

[pirxthepilot]

Thanks, Bruce! That makes sense and that's what I thought as well.