EDITED: the post below shows the problem more clear.
it takes 0.16 s to read 4 branches (colunms)
and 2400 s when 16 branches (columns) are read
Would anyone guess of the origin of such behavior
JuliaHEP:master ← JuliaHEP:branches-of-arrays
Here is a root script to produce the tree:
```c++
#include "TString.h"
#inclu… de "TFile.h"
#include "TTree.h"
#include <string>
#include "stdlib.h"
int main() {
TFile *f = new TFile("test_tree.root", "recreate");
TTree *t = new TTree("mytree", "mytree");
auto app = "XYZE";
const uint Np = 100;
//
double v[Np][4];
for (uint n=0; n<Np; n++)
for (uint i=0; i<4; i++)
t->Branch(TString::Format("p%d_%c",n, app[i]), &v[n][i]);
const int Nev = 10000;
for (uint i=0; i <= Nev; i++) {
for (uint n=0; n<Np; n++)
for (uint i=0; i<4; i++)
v[n][i] = (rand() % 100) / 100.0;
//
t->Fill();
}
t->Write();
f->Close();
return 0;
}
```
then, you can try (only `*_X` branches)
```julia
@time let
f = TFile(joinpath("test_tree.root"))
t = f["mytree"]
t.p1_X + t.p2_X
end
```
will take forever (11 sec.).
Surely faster to convert the whole tree and, then, access.
```
@btime let
f = TFile(joinpath("test_tree.root"))
t = f["mytree"][1:end]
t.p1_X + t.p2_X
end # 351.006 ms
```
Creating the three with only needed branches is the fastest option
```julia
@btime let
f = TFile(joinpath("test_tree.root"))
t = f["mytree"]
tT = getindex(t, eachindex(t), ["p1_X", "p2_X"])
#
tT.p1_X + tT.p2_X
end # 328.482 ms
```
The difference between the last two scales with the number of branches.
The file has 300_000 rows, the size of 10MB
Some more retails are below
the Particle is a simple mutable struct, the process function creates a named tuple of for particles
using UpROOT
using Parameters
#
@with_kw mutable struct Particle
E::Float64
px::Float64
py::Float64
pz::Float64
end
#
process(row) =
(pb=Particle(
px = row.pBeamX,
py = row.pBeamY,
pz = row.pBeamZ,
E = row.pBeamE),
pr = Particle(
px = row.pRecoilX,
py = row.pRecoilY,
pz = row.pRecoilZ,
E = row.pRecoilE),
pπ = Particle(
px = row.pPimX,
py = row.pPimY,
pz = row.pPimZ,
E = row.pPimE),
pη = Particle(
px = row.pEtaX,
py = row.pEtaY,
pz = row.pEtaZ,
E = row.pEtaE))
Pin @oschulz . Any ideas?
Just found a typo in the second execution:process.(tr) instead of process.(t). Then it takes 1s
What remains unclear why processing the tree directly is so much slower. It contains
Well, here is the shortest representation of the problem:
Conversion of the TTree (in disk) to Table (in memory) is done when getindex is called.
Probably best if we discuss this on your GibHub issue (https://github.com/JuliaHEP/UpROOT.jl/issues/12 ), in case we need to make changes to UpROOT.jl
2 Likes
