Converting RDataFrames to awkward array

[alig146]

Hi,

I’m trying to convert RDataFrame to awkward array and i seem to get a error with ak.from_rdataframe. Basically i have ROOT and Awkward downloaded via conda and when i run the following:

df3 = ROOT.RDataFrame("CollectionTree", "/global/u2/a/agarabag/pscratch/ditdau_samples/graviton.root")
npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", "EventInfoAuxDyn.mcEventWeights"), keep_order=True)

i get the below error:

File "/global/u2/a/agarabag/plotter_v5.py", line 534, in plot_branches
npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", "EventInfoAuxDyn.mcEventWeights"), keep_order=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_dispatch.py", line 39, in dispatch
gen_or_result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/operations/ak_from_rdataframe.py", line 56, in from_rdataframe
return _impl(rdf, columns, highlevel, behavior, with_name, offsets_type, keep_order)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/operations/ak_from_rdataframe.py", line 62, in _impl
import awkward._connect.rdataframe.from_rdataframe # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/ROOT/_facade.py", line 154, in _importhook
return _orig_ihook(name, *args, **kwds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/rdataframe/from_rdataframe.py", line 51, in
cppyy.add_include_path(
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy/init.py", line 221, in add_include_path
raise OSError("no such directory: %s" % path)
OSError: no such directory: /global/u2/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/header-only

This error occurred while calling

ak.from_rdataframe(
RDataFrame-instance
columns = ('DiTauJetsAuxDyn.ditau_pt', 'EventInfoAuxDyn.mcEventWeights')
keep_order = True
)

I wanted to know if anyone knows what could be causing this error? i have tried removing and re downloading awkward but that hasn’t helped. The “header-only” dir indeed does not exist but I’m not sure why. also awkward functions normally its only when calling ak.from_rdataframe that i get this error.

ROOT Version: 6.28.4
awkward version: 2.5.0

[ianna]

Hi Ali,The header-only is a part of awkward-cpp. Please check if it is installed. Thanks, Yana. Sent from my iPhone

[alig146]

Hi Yana,

yes, awkward-cpp is installed (version 26). also i tried to downgrade to 2.4 and got the same error and i also jumped to version 2.1.1 and the code is running now.

[alig146]

also I wanted to ask a bit unrelated questions. the code works with version 2.1.1 on rather small root files. when i try to run on root files > 1 GB i get "Segmentation fault". i assume this is a memory issue. does this mean my machine does not have enough memory or something more complicated? and also wanted to check that is this a known limit, that is with above some threshold ak.from_rdataframe doesn't work?

[ianna]

Hi Yana,

yes, awkward-cpp is installed (version 26). also i tried to downgrade to 2.4 and got the same error and i also jumped to version 2.1.1 and the code is running now.

Oh, it looks like a bug. Thanks for reporting it!

I am constructing a path to the header-only directory, but it looks like it is valid only in a developers installation:

awkward/src/awkward/_connect/rdataframe/from_rdataframe.py

Lines 51 to 59 in ae5923e

	cppyy.add_include_path(
	os.path.abspath(
	os.path.join(
	os.path.dirname(os.path.realpath(__file__)),
	os.path.pardir,
	"header-only",
	)
	)
	)

[agoose77]

This should work in production, even though it won't work with zipfile modules. Perhaps @alig146 could report what the following script yields?

#!/usr/bin/env python3
import awkward._connect.rdataframe.from_rdataframe
import os.path

search_path = os.path.abspath( 
   os.path.join( 
       os.path.dirname(os.path.realpath(awkward._connect.rdataframe.from_rdataframe.__file__)), 
       os.path.pardir, 
       "header-only", 
   ) 
) 
print(search_path)
print(os.listdir(search_path))

[agoose77]

Then, which parts of the path that is reported actually exist on your file system?

[alig146]

Hi I ran the script. and here are the print results

for awkward version 2.1.1:
first print:
/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/loader.py:90: UserWarning: No precompiled header available (/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/etc not writable); this may impact performance.
warnings.warn('No precompiled header available (%s); this may impact performance.' % msg)
/global/u2/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/header-only

second print:
['awkward']

for awkward version 2.5.0:
first print:
/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/loader.py:90: UserWarning: No precompiled header available (/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy_backend/etc not writable); this may impact performance.
warnings.warn('No precompiled header available (%s); this may impact performance.' % msg)
Traceback (most recent call last):
File "/global/u2/a/agarabag/test.py", line 2, in
import awkward._connect.rdataframe.from_rdataframe
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/rdataframe/from_rdataframe.py", line 51, in
cppyy.add_include_path(
File "/global/homes/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/cppyy/init.py", line 221, in add_include_path
raise OSError("no such directory: %s" % path)
OSError: no such directory: /global/u2/a/agarabag/.conda/envs/ditau/lib/python3.11/site-packages/awkward/_connect/header-only

In version 2.1.1 header-only directory exists while in 2.5.0 it does not.

[agoose77]

Ah @alig146 I think I'm figuring out what's happened here. Thanks!

[ianna]

also I wanted to ask a bit unrelated questions. the code works with version 2.1.1 on rather small root files. when i try to run on root files > 1 GB i get "Segmentation fault". i assume this is a memory issue. does this mean my machine does not have enough memory or something more complicated? and also wanted to check that is this a known limit, that is with above some threshold ak.from_rdataframe doesn't work?

I think, an insufficient memory is a valid guess. You can try to monitor it to check it's limit on a file size for your computer. It may depend on what else you are running. The stack trace could provide more detailed information about it.

[alig146]

when i run over > 1GB file which I think is actually not that large, i just get the Segmentation fault message and the script is terminated there is no other print (i assume this is what you meant by stack trace). also i'm running on perlmutter cluster now so i think the computing resources are fine. if you like to reproduce this i have put the root files, conda env, and script here: https://cernbox.cern.ch/s/BYiAkT0HMXxBCHI

[ianna]

when i run over > 1GB file which I think is actually not that large, i just get the Segmentation fault message and the script is terminated there is no other print (i assume this is what you meant by stack trace). also i'm running on perlmutter cluster now so i think the computing resources are fine. if you like to reproduce this i have put the root files, conda env, and script here: https://cernbox.cern.ch/s/BYiAkT0HMXxBCHI

Indeed, I've tried to run it on a local copy of your file - it doesn't take long at all - it's not that large:

In [1]: import ROOT

In [2]: import awkward as ak

In [3]: ROOT.ROOT.EnableImplicitMT(16)
Installed ROOT event loop hook.

In [4]: %%timeit
   ...: df3 = ROOT.RDataFrame("CollectionTree", "/Users/yana/Downloads/user.agarabag.34455039._000004.output.root")
   ...: 
   ...: 
The slowest run took 8.06 times longer than the fastest. This could mean that an intermediate result is being cached.
830 µs ± 1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

retrieving the columns takes a bit longer (the data is copied):

In [7]: %%timeit
   ...: npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", "EventInfoAuxDyn.mcEventWeights"), keep_order=True)
   ...: 
   ...: 
Warning in <TClass::Init>: no dictionary for class xAOD::EventFormat_v1 is available
Warning in <TClass::Init>: no dictionary for class ElementLinkBase is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TruthParticle_v1> > is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::EventInfo_v1> > is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TrackParticle_v1> > is available
Warning in <TClass::Init>: no dictionary for class xAOD::EventInfo_v1 is available
Warning in <TClass::Init>: no dictionary for class SG::AuxElement is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxElement is available
Warning in <TClass::Init>: no dictionary for class xAOD::EventAuxInfo_v2 is available
Warning in <TClass::Init>: no dictionary for class xAOD::AuxInfoBase is available
Warning in <TClass::Init>: no dictionary for class xAOD::AuxContainerBase is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TruthVertex_v1> > is available
Warning in <TClass::Init>: no dictionary for class xAOD::TruthParticleAuxContainer_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::TruthParticle_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::TauJet_v3 is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::Jet_v1> > is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::Vertex_v1> > is available
Warning in <TClass::Init>: no dictionary for class xAOD::DiTauJetAuxContainer_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::DiTauJet_v1 is available
Warning in <TClass::Init>: no dictionary for class xAOD::ShallowAuxContainer is available
Warning in <TClass::Init>: no dictionary for class DataLink<SG::IConstAuxStore> is available
Warning in <TClass::Init>: no dictionary for class ElementLink<DataVector<xAOD::TruthEventBase_v1> > is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxStore is available
Warning in <TClass::Init>: no dictionary for class SG::IConstAuxStore is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxStoreIO is available
Warning in <TClass::Init>: no dictionary for class SG::IAuxStoreHolder is available
Warning in <TClass::Init>: no dictionary for class xAOD::IParticle is available
Warning in <TClass::Init>: no dictionary for class DataLinkBase is available
5.21 s ± 90.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

and it produces some result:

In [10]: print(ak.flatten(npy3["DiTauJetsAuxDyn.ditau_pt"]))
[0, 2.83e+04, 2.17e+04, 4.03e+04, 0, ..., 0, 2.29e+04, 5.45e+04, 2.6e+05]

I'm using different versions though:

>>> ROOT.__version__
'6.28/04'
>>> ak.__version__
'2.4.5'
>>> 

I'll try to do the same with awkward 2.5.1.

[ianna]

@alig146 - it is possible that the failing jobs produce a log file somewhere on the cluster. Please, check with your devops. Alternatively, I could suggest to: please, try run it in single thread, run the script with just opening the file, try extracting one column, try to apply a filter before extracting the columns.

[alig146]

hi, thank you for testing the script and the suggestions, i will also try to reach out to the devops. regarding your suggestions i tried your suggestions and i only got it to work when extracting just one column as shown below:
npy3 = ak.from_rdataframe(df3, columns=("DiTauJetsAuxDyn.ditau_pt", ), keep_order=True)
then i tried other variables and they also worked (with single column), i also tried multiple columns of other variables and it worked except when i included "EventInfoAuxDyn.mcEventWeights" (even with single column) so i guess it doesn't like this variable for some reason. but its strange that it worked for you. unfortunately i can't try version 2.4.5 because of the header-only error.

[alig146]

i also got it to work with newer version. however, when i started running over all my root files i again ran into the Segfault with files > 4 GB. I have put one of these files in Cernbox (user.agarabag.34455039._000003.output.root). If you have time could you please see if you can run this one as well (i have also tried running with 2.5.1).