ArrowInvalid: Length spanned by list offsets (200000) larger than values array (length 4662)

[gsaha009]

Hello Awkward Experts,

I have been trying to save an array in arrow which has quite a nested structure
The array looks like (one field from the array)

filed_1 = {'pt' : [ [[], []], [[], [21.1, 23.2]], [[29.1], [21.9, 43.1, 51.1]], ... ]}

Then I used

ak.enforce_type(ak.values_astype(ak.nan_to_num(field_1, 0.0), "float32"),  "var * var * float32")

It used to work before, but now I have upgraded the awkward version from 2.6.6 to 2.7.1, and it gets failed at

ak.to_arrow(field_1)

with the following error:

ArrowInvalid: Length spanned by list offsets (200000) larger than values array (length 4662)

This error occurred while calling

    ak.to_arrow(
        <Array [[[], []], [[], []], ..., [], []] type='100000 * var * var *...'>
    )

I am aware of the nested structure of the array, but if I convert the field into list and then to ak.Array again, it works. Any idea would be very helpful.

Thanks

Best,
Gourab

[ianna]

@gsaha009 - thanks for reporting the issue. This looks like a bug. I will have a look. Thanks!