Performance issue with `Tables.columns` fallback functions at 1st run with `Tables.Schema = nothing` and `Base.IteratorSize = SizeUnknown()`

[mathieu17g]

Hello,

I'm not aware if any user implementing the Tables interface and relying on fallback functions with no Schema has already encountered a performance issue at compilation, but here is below a particular case encountered in ArchGDAL.jl

Using the Tables.jl interface on ArchGDAL.jl layer objects, @visr stumbled on a performance issue with Tables.columns fallback functions at 1st run, with Tables.Schema = nothing and Base.IteratorSize = SizeUnknown(). The performance bottleneck at first run is located in Tables.add_or_widen! (seen when profiling). Probably around the columm allocation machinery based on Tuple.

There is no problem when providing detailed types in the Schema. Unfortunately, we cannot know the tightest schema for sure from a layer feature definition.

The problem can be reproduced with sequence below: ~170s at first run for a 10 row table with more than 300 columns

Download BasinATLAS_v10_lev01.zip and decompress it in ./BasinATLAS_v10_lev01/

julia> import ArchGDAL as AG

julia> using DataFrames

julia> using Profile

julia> @time @profile DataFrame(AG.getlayer(AG.read("BasinATLAS_v10_lev01")))
172.879812 seconds (53.38 M allocations: 2.992 GiB, 0.44% gc time, 99.38% compilation time)
10×295 DataFrame
 Row │                            HYBAS_ID    NEXT_DOWN  NEXT_SINK   MAIN_BAS    DIST_SINK  DIST_MAIN  SUB_AREA   UP_AREA    PFAF_ID  EN ⋯
     │ IGeometr…                  Int64       Int64      Int64       Int64       Float64    Float64    Float64    Float64    Int64    In ⋯
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ Geometry: wkbMultiPolygon  1010000010          0  1010000010  1010000010        0.0        0.0  2.99531e7  2.99531e7        1     ⋯
   2 │ Geometry: wkbMultiPolygon  2010000010          0  2010000010  2010000010        0.0        0.0  1.78589e7  1.78589e7        2
   3 │ Geometry: wkbMultiPolygon  3010000010          0  3010000010  3010000010        0.0        0.0  1.29491e7  1.29491e7        3
   4 │ Geometry: wkbMultiPolygon  4010000010          0  4010000010  4010000010        0.0        0.0  2.08274e7  2.08274e7        4
   5 │ Geometry: wkbMultiPolygon  5010000010          0  5010000010  5010000010        0.0        0.0  1.08038e7  1.08038e7        5     ⋯
   6 │ Geometry: wkbMultiPolygon  6010000010          0  6010000010  6010000010        0.0        0.0  1.78535e7  1.78535e7        6
   7 │ Geometry: wkbMultiPolygon  7010000010          0  7010000010  7010000010        0.0        0.0  1.59146e7  1.59146e7        7
   8 │ Geometry: wkbMultiPolygon  8010000010          0  8010000010  8010000010        0.0        0.0  6.19709e6  6.19709e6        8
   9 │ Geometry: wkbMultiPolygon  8010020760          0  8010020760  8010020760        0.0        0.0  1.17002e5  1.17002e5        3     ⋯
  10 │ Geometry: wkbMultiPolygon  9010000010          0  9010000010  9010000010        0.0        0.0  2.14672e6  2.14672e6        9
                                                                                                                       285 columns omitted

julia> Profile.print(;mincount=5000)
Overhead ╎ [+additional indent] Count File:Line; Function
=========================================================
      ╎123273 @Base/client.jl:495; _start()
      ╎ 123273 @Base/client.jl:309; exec_options(opts::Base.JLOptions)
      ╎  123273 @Base/client.jl:379; run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, hi...
      ╎   123273 @Base/essentials.jl:714; invokelatest
      ╎    123273 @Base/essentials.jl:716; #invokelatest#2
      ╎     123273 @Base/client.jl:394; (::Base.var"#930#932"{Bool, Bool, Bool})(REPL::Module)
      ╎    ╎ 123273 ...hare/julia/stdlib/v1.7/REPL/src/REPL.jl:349; run_repl(repl::REPL.AbstractREPL, consumer::Any)
      ╎    ╎  123273 ...are/julia/stdlib/v1.7/REPL/src/REPL.jl:362; run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on...
      ╎    ╎   123273 ...are/julia/stdlib/v1.7/REPL/src/REPL.jl:229; start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
      ╎    ╎    123273 ...re/julia/stdlib/v1.7/REPL/src/REPL.jl:244; repl_backend_loop(backend::REPL.REPLBackend)
      ╎    ╎     123273 ...re/julia/stdlib/v1.7/REPL/src/REPL.jl:150; eval_user_input(ast::Any, backend::REPL.REPLBackend)
      ╎    ╎    ╎ 123273 @Base/boot.jl:373; eval
      ╎    ╎    ╎  123273 REPL[4]:0; top-level scope
      ╎    ╎    ╎   123273 @Base/timing.jl:220; top-level scope
      ╎    ╎    ╎    123273 ...a/stdlib/v1.7/Profile/src/Profile.jl:28; macro expansion
      ╎    ╎    ╎     123273 @DataFrames/src/other/tables.jl:49; DataFrame
      ╎    ╎    ╎    ╎ 123073 @DataFrames/src/other/tables.jl:58; DataFrame(x::ArchGDAL.IFeatureLayer; copycols::Nothing)
   147╎    ╎    ╎    ╎  123073 @Tables/src/fallbacks.jl:253; columns
   192╎    ╎    ╎    ╎   122879 @Tables/src/fallbacks.jl:217; buildcolumns
      ╎    ╎    ╎    ╎    122029 @Tables/src/fallbacks.jl:187; _buildcolumns(rowitr::ArchGDAL.IFeatureLayer, row::A...
    58╎    ╎    ╎    ╎     122029 @Tables/src/utils.jl:132; eachcolumns
101464╎    ╎    ╎    ╎    ╎ 121929 @Tables/src/fallbacks.jl:153; add_or_widen!(val::ArchGDAL.IGeometry{ArchGDAL.wkbM...
      ╎    ╎    ╎    ╎    ╎  5732   @Base/compiler/typeinfer.jl:938; typeinf_ext_toplevel(mi::Core.MethodInstance, worl...
      ╎    ╎    ╎    ╎    ╎   5731   @Base/compiler/typeinfer.jl:942; typeinf_ext_toplevel(interp::Core.Compiler.Native...
      ╎    ╎    ╎    ╎    ╎    5721   @Base/compiler/typeinfer.jl:909; typeinf_ext(interp::Core.Compiler.NativeInterpret...
      ╎    ╎    ╎    ╎    ╎     5721   @Base/compiler/typeinfer.jl:209; typeinf(interp::Core.Compiler.NativeInterpreter,...
      ╎    ╎    ╎    ╎    ╎    ╎ 5695   @Base/compiler/typeinfer.jl:226; _typeinf(interp::Core.Compiler.NativeInterpreter...
      ╎    ╎    ╎    ╎    ╎    ╎  5695   ...iler/abstractinterpretation.jl:2014; typeinf_nocycle(interp::Core.Compiler.NativeIn...
      ╎    ╎    ╎    ╎    ╎    ╎   5693   ...ler/abstractinterpretation.jl:1918; typeinf_local(interp::Core.Compiler.NativeInte...
      ╎    ╎    ╎    ╎    ╎    ╎    5693   ...ler/abstractinterpretation.jl:1534; abstract_eval_statement(interp::Core.Compiler...
      ╎    ╎    ╎    ╎    ╎    ╎     5693   ...ler/abstractinterpretation.jl:1382; abstract_call(interp::Core.Compiler.NativeInt...
      ╎    ╎    ╎    ╎    ╎    ╎    ╎ 5693   ...er/abstractinterpretation.jl:1397; abstract_call(interp::Core.Compiler.NativeIn...
      ╎    ╎    ╎    ╎    ╎    ╎    ╎  5677   ...er/abstractinterpretation.jl:1342; abstract_call_known(interp::Core.Compiler.N...
      ╎    ╎    ╎    ╎    ╎    ╎    ╎   5421   ...er/abstractinterpretation.jl:105; abstract_call_gf_by_type(interp::Core.Compi...
      ╎    ╎    ╎    ╎    ╎    ╎    ╎    5419   ...r/abstractinterpretation.jl:504; abstract_call_method(interp::Core.Compiler....
      ╎    ╎    ╎    ╎    ╎    ╎    ╎     5408   @Base/compiler/typeinfer.jl:823; typeinf_edge
      ╎    ╎    ╎    ╎    ╎    ╎    ╎    ╎ 5408   @Base/compiler/typeinfer.jl:209; typeinf(interp::Core.Compiler.NativeInterp...
     1╎    ╎    ╎    ╎    ╎  14732  @Tables/src/fallbacks.jl:143; replacex(t::NTuple{295, Tables.EmptyVector}, col::...
      ╎    ╎    ╎    ╎    ╎   14731  @Base/ntuple.jl:19; ntuple
     8╎    ╎    ╎    ╎    ╎    14731  @Base/ntuple.jl:37; _ntuple(f::Tables.var"#41#42"{NTuple{295, Tables....
  6228╎    ╎    ╎    ╎    ╎     14717  @Base/array.jl:734; collect(itr::Base.Generator{UnitRange{Int64}, Ba...
   131╎    ╎    ╎    ╎    ╎    ╎ 7799   @Base/array.jl:760; collect_to_with_first!(dest::Vector{Vector{ArchG...
  6890╎    ╎    ╎    ╎    ╎    ╎  7653   @Base/array.jl:790; collect_to!(dest::Vector{Vector{ArchGDAL.IGeome...
Total snapshots: 125792

julia> 

At the 2nd run the performance is normal: 1.5s (x100 vs 1st run)

julia> @time @profile DataFrame(AG.getlayer(AG.read("BasinATLAS_v10_lev01")))
  1.461657 seconds (246.65 k allocations: 210.206 MiB, 31.72% gc time, 1.47% compilation time)
10×295 DataFrame
 Row │                            HYBAS_ID    NEXT_DOWN  NEXT_SINK   MAIN_BAS    DIST_SINK  DIST_MAIN  SUB_AREA   UP_AREA    PFAF_ID  EN ⋯
     │ IGeometr…                  Int64       Int64      Int64       Int64       Float64    Float64    Float64    Float64    Int64    In ⋯
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ Geometry: wkbMultiPolygon  1010000010          0  1010000010  1010000010        0.0        0.0  2.99531e7  2.99531e7        1     ⋯
   2 │ Geometry: wkbMultiPolygon  2010000010          0  2010000010  2010000010        0.0        0.0  1.78589e7  1.78589e7        2
   3 │ Geometry: wkbMultiPolygon  3010000010          0  3010000010  3010000010        0.0        0.0  1.29491e7  1.29491e7        3
   4 │ Geometry: wkbMultiPolygon  4010000010          0  4010000010  4010000010        0.0        0.0  2.08274e7  2.08274e7        4
   5 │ Geometry: wkbMultiPolygon  5010000010          0  5010000010  5010000010        0.0        0.0  1.08038e7  1.08038e7        5     ⋯
   6 │ Geometry: wkbMultiPolygon  6010000010          0  6010000010  6010000010        0.0        0.0  1.78535e7  1.78535e7        6
   7 │ Geometry: wkbMultiPolygon  7010000010          0  7010000010  7010000010        0.0        0.0  1.59146e7  1.59146e7        7
   8 │ Geometry: wkbMultiPolygon  8010000010          0  8010000010  8010000010        0.0        0.0  6.19709e6  6.19709e6        8
   9 │ Geometry: wkbMultiPolygon  8010020760          0  8010020760  8010020760        0.0        0.0  1.17002e5  1.17002e5        3     ⋯
  10 │ Geometry: wkbMultiPolygon  9010000010          0  9010000010  9010000010        0.0        0.0  2.14672e6  2.14672e6        9
      

I got around this issue in PR yeesian/ArchGDAL.jl#266 in which Tables.columns has been implemented primarily to address other performance issues more directly linked to GDAL specific C interface constraints.

I used a mutable grid: Vector{Vector{T} where T} with wide initial types instead of NamedTuple iteratively extended on column length and column types.

I tightened the column types a posteriori.

EDIT 1: Note that the iterator's size is advertised as SizeUnknown() because for some sources, GDAL drivers may reset the iterator if length retrieval is forced. Therefore length is not forced to avoid any side effect and can stay unknown.
Forcing the length retrieval was breaking the parsing done by Tables.buildcolumns with Schema = nothing for such GDAL drivers (ex. "GML" driver, see comment yeesian/ArchGDAL.jl#226 (comment) for further details if necessary)

[bkamins]

Maybe something can be done about it in Tables.jl. But for now the typical solution to such a problem is, as you note to provide a custom Tables.columns implementation. E.g. in DataFrames.jl we use the following definition:

Tables.columns(df::AbstractDataFrame) = eachcol(df)

that ensures that there are no compilation problems even for tables having 10000 and more columns.

[quinnj]

Thanks for the report; I think we can certainly do better here, but as noted, by relying on the fallback Tables.columns, you're inherently asking for a NamedTuple of Vectors, which will have to construct a potentially very wide NamedTuple at some point in process. We could use a vector of vectors in the intermediate code, but we'd have to profile if that makes a big difference if we still have to construct the NamedTuple at the end.

Alternatively, it'd probably be worth profiling Tables.dictcolumntable for your case which uses a Dict as the internal machinery and shouldn't have anywhere near the same compilation cost.

[mathieu17g]

Maybe there are some side effects, but if you inline replacex in add_or_widen! the issue disappear: 1st run = ~x4 2nd run instead of x100

By inlining, I mean replacing

Tables.jl/src/fallbacks.jl

Line 153 in 5e62ab2

updated[] = replacex(updated[], col, new)

by

        updated[] = ntuple(i->i == col ? new : updated[][i], length(updated[]))

Inlining by adding @inline to replacex does not work

julia> import ArchGDAL as AG; using DataFrames; using Profile

julia> @time @profile DataFrame(AG.getlayer(AG.read("tmp/BasinATLAS_v10_lev01")))
  5.613708 seconds (6.84 M allocations: 342.865 MiB, 2.79% gc time, 79.85% compilation time)
10×295 DataFrame
 Row │                            HYBAS_ID    NEXT_DOWN  NEXT_SINK   MAIN_BAS    DIST_SINK  DIST_MAIN   ⋯
     │ IGeometr…                  Int64       Int64      Int64       Int64       Float64    Float64     ⋯
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ Geometry: wkbMultiPolygon  1010000010          0  1010000010  1010000010        0.0        0.0   ⋯
   2 │ Geometry: wkbMultiPolygon  2010000010          0  2010000010  2010000010        0.0        0.0
   3 │ Geometry: wkbMultiPolygon  3010000010          0  3010000010  3010000010        0.0        0.0
   4 │ Geometry: wkbMultiPolygon  4010000010          0  4010000010  4010000010        0.0        0.0
   5 │ Geometry: wkbMultiPolygon  5010000010          0  5010000010  5010000010        0.0        0.0   ⋯
   6 │ Geometry: wkbMultiPolygon  6010000010          0  6010000010  6010000010        0.0        0.0
   7 │ Geometry: wkbMultiPolygon  7010000010          0  7010000010  7010000010        0.0        0.0
   8 │ Geometry: wkbMultiPolygon  8010000010          0  8010000010  8010000010        0.0        0.0
   9 │ Geometry: wkbMultiPolygon  8010020760          0  8010020760  8010020760        0.0        0.0   ⋯
  10 │ Geometry: wkbMultiPolygon  9010000010          0  9010000010  9010000010        0.0        0.0
                                                                                      288 columns omitted

julia> 

EDIT 1: as a side effect we loose the "barrier function" gain at second run. In my case it is ~20% performance loss. But in my case there are few widening occurrences. The performance loss may be bigger for cases where there are many widening occurrences.

[mathieu17g]

Alternatively, it'd probably be worth profiling Tables.dictcolumntable for your case which uses a Dict as the internal machinery and shouldn't have anywhere near the same compilation cost.

Indeed, it is far better as can be seen below

julia> import ArchGDAL as AG; using DataFrames; using Profile; using BenchmarkTools; using Tables
       Profile.init(n = 10^9, delay = 0.0001)
[ Info: Precompiling ArchGDAL [c9ce4bd3-c3d5-55b8-8973-c0e20141b8c3]
[ Info: Precompiling DataFrames [a93c6f00-e57d-5684-b7b6-d8193f3e46c0]

julia> @time DataFrame(Tables.dictcolumntable(AG.getlayer(AG.read("tmp/BasinATLAS_v10_lev01"))))
  1.438461 seconds (883.37 k allocations: 47.894 MiB, 9.64% gc time, 47.82% compilation time)
10×295 DataFrame
 Row │ glc_pc_s06  lit_cl_smj  pre_mm_syr  nli_ix_uav  glc_pc_s20  wet_pc_s09  tmp_dc_s08  glc_pc_u22   ⋯
     │ Int32       Int32       Int32       Int32       Int32       Int32       Int32       Int32        ⋯
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1 │          0           1         656        6270           1           1         251         100   ⋯
   2 │          7           5         499        6279           2           1         197         100
   3 │          5           3         377           0           2           0         109           0
   4 │          1           1         739           0           1           1         207           0
   5 │          1           1        1088           0           1           1         175           0   ⋯
   6 │          0           1        1508          14           1           0         194           0
   7 │          6           3         789        2925           6           0         189           0
   8 │          2           5         295           0           7           0          74           0
   9 │          0           5         391         858           8           0          53           0   ⋯
  10 │          0          16         681           0           0           0         -31           0
                                                                                      287 columns omitted

julia> @time DataFrame(Tables.dictcolumntable(AG.getlayer(AG.read("tmp/BasinATLAS_v10_lev01"))))
  0.724144 seconds (19.41 k allocations: 748.625 KiB)
10×295 DataFrame
 Row │ glc_pc_s06  lit_cl_smj  pre_mm_syr  nli_ix_uav  glc_pc_s20  wet_pc_s09  tmp_dc_s08  glc_pc_u22   ⋯
     │ Int32       Int32       Int32       Int32       Int32       Int32       Int32       Int32        ⋯
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1 │          0           1         656        6270           1           1         251         100   ⋯
   2 │          7           5         499        6279           2           1         197         100
   3 │          5           3         377           0           2           0         109           0
   4 │          1           1         739           0           1           1         207           0
   5 │          1           1        1088           0           1           1         175           0   ⋯
   6 │          0           1        1508          14           1           0         194           0
   7 │          6           3         789        2925           6           0         189           0
   8 │          2           5         295           0           7           0          74           0
   9 │          0           5         391         858           8           0          53           0   ⋯
  10 │          0          16         681           0           0           0         -31           0
                                                                                      287 columns omitted

julia>

Unfortunately, it puts the column in a weird order. But I will look into it, in case it case it can do better than Vector{Vector{T} where T} in ArchGDAL implementation of Tables.columns

As for this issue, I had not seen the hint to use Tables.dictcolumntable for sources without any Schema. It is a good solution for those not keen to implement their own Tables.columns function.

Shall I close the issue ?

[bkamins]

@quinnj - maybe use OrderedDict in Tables.dictcolumntable?

[mathieu17g]

@bkamins I have tested with OrderedDict, the columns are put back in the right order without visible performance impact

julia> import ArchGDAL as AG; using DataFrames; using Profile; using BenchmarkTools; using Tables
       Profile.init(n = 10^9, delay = 0.0001)

julia> @time DataFrame(Tables.dictcolumntable(AG.getlayer(AG.read("tmp/BasinATLAS_v10_lev01"))))
  1.294441 seconds (870.91 k allocations: 47.437 MiB, 1.48% gc time, 40.37% compilation time)
10×295 DataFrame
 Row │                            HYBAS_ID    NEXT_DOWN  NEXT_SINK   MAIN_BAS    DIST_SINK  DIST_MAIN   ⋯
     │ IGeometr…                  Int64       Int64      Int64       Int64       Float64    Float64     ⋯
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ Geometry: wkbMultiPolygon  1010000010          0  1010000010  1010000010        0.0        0.0   ⋯
   2 │ Geometry: wkbMultiPolygon  2010000010          0  2010000010  2010000010        0.0        0.0
   3 │ Geometry: wkbMultiPolygon  3010000010          0  3010000010  3010000010        0.0        0.0
   4 │ Geometry: wkbMultiPolygon  4010000010          0  4010000010  4010000010        0.0        0.0
   5 │ Geometry: wkbMultiPolygon  5010000010          0  5010000010  5010000010        0.0        0.0   ⋯
   6 │ Geometry: wkbMultiPolygon  6010000010          0  6010000010  6010000010        0.0        0.0
   7 │ Geometry: wkbMultiPolygon  7010000010          0  7010000010  7010000010        0.0        0.0
   8 │ Geometry: wkbMultiPolygon  8010000010          0  8010000010  8010000010        0.0        0.0
   9 │ Geometry: wkbMultiPolygon  8010020760          0  8010020760  8010020760        0.0        0.0   ⋯
  10 │ Geometry: wkbMultiPolygon  9010000010          0  9010000010  9010000010        0.0        0.0
                                                                                      288 columns omitted

julia> @time DataFrame(Tables.dictcolumntable(AG.getlayer(AG.read("tmp/BasinATLAS_v10_lev01"))))
  0.724550 seconds (21.18 k allocations: 835.625 KiB)
10×295 DataFrame
 Row │                            HYBAS_ID    NEXT_DOWN  NEXT_SINK   MAIN_BAS    DIST_SINK  DIST_MAIN   ⋯
     │ IGeometr…                  Int64       Int64      Int64       Int64       Float64    Float64     ⋯
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ Geometry: wkbMultiPolygon  1010000010          0  1010000010  1010000010        0.0        0.0   ⋯
   2 │ Geometry: wkbMultiPolygon  2010000010          0  2010000010  2010000010        0.0        0.0
   3 │ Geometry: wkbMultiPolygon  3010000010          0  3010000010  3010000010        0.0        0.0
   4 │ Geometry: wkbMultiPolygon  4010000010          0  4010000010  4010000010        0.0        0.0
   5 │ Geometry: wkbMultiPolygon  5010000010          0  5010000010  5010000010        0.0        0.0   ⋯
   6 │ Geometry: wkbMultiPolygon  6010000010          0  6010000010  6010000010        0.0        0.0
   7 │ Geometry: wkbMultiPolygon  7010000010          0  7010000010  7010000010        0.0        0.0
   8 │ Geometry: wkbMultiPolygon  8010000010          0  8010000010  8010000010        0.0        0.0
   9 │ Geometry: wkbMultiPolygon  8010020760          0  8010020760  8010020760        0.0        0.0   ⋯
  10 │ Geometry: wkbMultiPolygon  9010000010          0  9010000010  9010000010        0.0        0.0
                                                                                      288 columns omitted

julia> 

[mathieu17g]

Coming back to the risk of performance loss of inlining replacex,

EDIT 1: as a side effect we loose the "barrier function" gain at second run. In my case it is ~20% performance loss. But in my case there are few widening occurrences. The performance loss may be bigger for cases where there are many widening occurrences.

I have tested it along with Tables.dictcolumntable on a bigger version of the test file provided above. It has the same structure but more data, 57k rows vs 10. Here are the results:

Current master Tables.columns with replacex function used in add_or_widen! :

• 1st run: 183.8 s
• 2nd run: 35.0 s

Current master Tables.columns with replacex inlined in add_or_widen! as described in #273 (comment)

• 1st run: 41.7 s
• 2nd run: 35.3 s

Current master Tables.dictcolumntable with Dict replaced by OrdererdDict:

• 1st run: 49.0 s
• 2nd run: 47.9 s

So all in all, the cheapest and more effective way to solve this issue within Tables.jl would be to inline replacex as described in #273 (comment)
It solves the compilation overhead for Schema-less sources, without any overhead once compilation done provided there is a significant number of rows (10 row for which I had a 20% delay overhead, is not really representative of common usage, I guess)

Regarding ArchGDAL.jl special case, I will stick to Vector{Vector{T} where T} in ArchGDAL's own implementation of Tables.columns which includes other optimisations specific to GDAL (resulting in a performance > x5 faster than the fastest fallback solution above for the same file). And concurrent file access could still be implemented later as in CSV.jl for further perf enhancement.

EDIT 1: 2nd runs execution time were calculated with @btime ... seconds=600

[mathieu17g]

Shall I close the issue after the merge of PR #275, or something else has to be done on the topic ?

[bkamins]

I think it is OK to be closed.