Skip to content

Chunk size is ignored when EOF is not a new line using ReadableStream #931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rcambrj opened this issue Apr 20, 2022 · 1 comment
Open

Comments

@rcambrj
Copy link

rcambrj commented Apr 20, 2022

When using a chunk size bigger than the ReadableStream, the last line is ignored in the first chunk and a second chunk is always provided.

given:

title,name
test title 01,test name 01
last line,last line

(without a new line at EOF)

running:

var readStream = fs.createReadStream(__dirname + '/sample.csv', 'utf8');
Papa.parse(readStream, {
	header: false,
	chunkSize: 4000000000,
	chunk: function(item) {
		console.log('>>>> chunk ' + JSON.stringify(item.data));
	},
});

will output:

>>>> chunk [["title","name"],["test title 01","test name 01"]]
>>>> chunk [["last line","last line"]]

This seems to be related to ignoreLastRow which was introduced in #135

Is this expected? Surely in a small enough file, there should be only one chunk.

@DavidCockerill
Copy link

I am seeing this also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants