Missing Characters after Reformat

[njascgil]

A strange issue: if I delete a line then immediately proceed to reformat the CSV file (with "Align vertically"), the first character of the line below the one deleted disappears.
If, on the other hand, I save the file before reformatting, everything appears to be fine.

This happens to me in this file, for instance:
courseid_319_participants-01.csv

[BdR76]

Thanks for posting this issue, I see the same thing and the steps to reproduce this are:

1. Open the CSV Lint window
2. Open the test data file
3. Select the first data row (starting with "João,Barreiras..")
4. Delete the entire row (including the newline)
5. Quickly press the CSV Lint button Reformat
6. Reformat to "Align vertically" or "Tab-separated" etc.

The result is that it cuts off the first character of the next line Filipe,Brito.. to just ilipe,Brito... The exact Reformat options don't really matter, and the same issue would happen if you select Sort or Add column instead of Reformat. Basically it happens when CSV Lint has to rebuild the data file in some way.

Like you said, if you save the file before reformatting it doesn't happen. But also, when you wait like 5 seconds after deleting the line, it also doesn't happen (!?).

It's very strange and I have seen this before. It has to do with Notepad++ caching the textfile data and it somehow ending up not being synchronised after you edit text. I'll have to look into this, but it's a long-standing issue and I'm not sure what I can do about this.

[BdR76]

Just to make clear what the issue is, see screenshots below, so

A) Open the file
B) make one edit, like change "test123@my." to "test12@my."
C) quickly open Reformat dialog and press OK, like within 2 seconds

Now one extra character after the edited part is missing, so "test12@my." becomes "test12my."

It always seems to happen at the point of where the edit is, and it only happens when you do it very quickly, like within 2 seconds. In other words, when you make the edit, open the Reformat dialog, wait 5 seconds, then press OK, then it doesn't happen. See also discussion here

[njascgil]

In my case, I don't even have to do it quickly. I tried waiting for 10+ seconds and it always missed some character.
Also, I can confirm that the cursor position does not matter, only the location of the previous edit.

[mpheath]

@BdR76

It always seems to happen at the point of where the edit is ...

Where the gap is.

I suspect this part of the code might be causing the issue and may need debugging.

CSVLint/CSVLintNppPlugin/PluginInfrastructure/ScintillaStreams.cs

Lines 45 to 60 in 7e2c409

	// If there is a gap we create two streams - one for the first part and one for the second, and concatenate them
	return ConcatenatingStream.FromFactoryFunctions(false, () =>
	{
	var rangePtr = doc.GetRangePointer(0, gap);
	unsafe
	{
	return new UnmanagedMemoryStream((byte*) rangePtr.ToPointer(), gap, gap, FileAccess.Read);
	}
	}, () =>
	{
	var rangePtr = doc.GetRangePointer(gap+1, length);
	unsafe
	{
	return new UnmanagedMemoryStream((byte*)rangePtr.ToPointer(), length-gap, length-gap, FileAccess.Read);
	}
	});

The concept of directly reading the document buffer from start to the gap and then reading gap + 1 to the end to concatenate into 1 stream might be a risk if the user is actively editing at that time of being read.

https://www.scintilla.org/ScintillaDoc.html#SCI_GETGAPPOSITION

SCI_GETGAPPOSITION returns the current gap position. This is a hint that applications can use to avoid calling SCI_GETRANGEPOINTER with a range that contains the gap and consequent costs of moving the gap.

This quote contains a hint.