File "complete"

[crh3675]

When we use this library (which is great), we still have to add some code to fs.stat the file once an inotify event occurs.

We must do this because larger files when copied come across the TCP pipe in smaller buffers until the file is completely written. We created some timeout events to make this work. Perhaps some attention can be put into addressing the idea of creating a custom EventEmitter or something of that nature when a file is completely written.

[yuanchuan]

Yes that would be quite useful, thanks for your suggestion :)

[victusfate]

for inspiration on polling, see here https://github.com/paulmillr/chokidar/blob/master/index.js#L265
ideally non-polling would be best, not sure how inotify delivers in_close events

[crh3675]

Just to note, we pass about 500,000 files through this library about 4 times a day (within 1 hour) with our custom timer check and it works flawlessly. Basically, stat the file when it comes in, set a timer for 500ms and stat again, if the size matches, presumedly the file is completely written. It hasn't failed yet. It may not be the best approach but it works for us

[crizstian]

@crh3675 hey could yo provide an example of you're aproach ?

[sandman45]

I did something similar but just did a recursive function (used timeouts initially) but decided against it because I was able to check file.size and the previous size and its been working great since.
Having file complete event would be awesome then I dont need to do this code.

Here is my example

const checkFile = function (info, previousSize){
    fs.stat(info.path, (err, fileInfo) => {
        if (err === null) {
            if (fileInfo.size === previousSize && fileInfo.size > 0) {
                uploadFile(info, false);
            } else {
                checkFile(info, fileInfo.size);
            }
        } else {
            console.log(`File not found ${err}`);
        }
    });
};