Support non-UTF-8 encodings and fail gracefully with unexpected encoding #7
Description
A customer was trying to import a Windows-1252 encoded document and the parser gets stuck in a loop when finding a bad encoding. For privacy reasons I will not post the particular document.
Two parts:
- At the very least, we should gracefully handle documents with unexpected encodings so that client apps don't hang and can display an error.
- Ideally support multiple encodings. Although we may expect modern CSV to be UTF-8 encoded, the whole idea of the CSV Dialect is interoperability with existing documents in their various flavors.