Querying Contents Of The Loaded Csv File

[bhreinb]

Hi there,

I'm looking at using a CSV library for loading and querying aspects of the loaded CSV file. I believe you take care of the first aspect very well in that your library is very performant compared to other libraries for example Apache Commons Csv hence why I came across your library. I wonder though do you have a solution to validate aspects of the loaded CSV file.

I don't want to validate all the loaded csv file against a baseline file as that would be very inefficient. The analogy i'd present would be for XML|JSON file types I can validate aspects of that file using Xpath|Jsonpath so I'm wondering whether your library has a possible solution to that and if not maybe you may have an idea how this could be solved? There isn't a native path syntax for csv so I'm wondering what could be the best possible approaches in the here and now? Maybe a sql syntax of sorts to validate Column 2 | Row 4 or Column Name | Row 4 perhaps? Appreciate any thoughts on the above :)

[osiegmar]

Hey @bhreinb,

a custom CallbackHandler could be used to efficiently load only data that is relevant for your process. The Custom callback handler example shows how to implement such a handler. Basically you only have to implement a class that extends CsvCallbackHandler and inject this custom callback handler to one of CsvReader.builder().build() methods. It's questionable if the effort is worth the performance gain, as the data needs to be parsed anyway, but that might be different in your situation.

FastCSV itself does currently not have any built-in mechanism to query/parse only specific fields. To query only specific records, the IndexedCsvReader could be used. An example for this could be found at https://fastcsv.org/guides/examples/indexed-read/.

Best, Oliver