# Tables.jl, UpROOT.jl and the speed

**URL:** https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737
**Category:** General Usage
**Created:** [August 11, 2020, 9:08am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737 "2020-08-11T09:08:05Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)
#### Post date: [August 11, 2020, 9:08am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/1 "2020-08-11T09:08:05Z")

</div>

EDITED: [the post below](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/4) shows the problem more clear.

it takes 0.16 s to read 4 branches (colunms)

 ![image](https://global.discourse-cdn.com/julialang/original/3X/9/2/92807ee8a4291fc3019267f0e7bbcc51fec7f4e8.png)

and 2400 s when 16 branches (columns) are read  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/7/4/740ff2c87b1339688a84fcc8992215c1f67b2326.png)

Would anyone guess of the origin of such behavior  
I should be able to provide the root file for testing in spirit of that

> <https://github.com/JuliaHEP/UpROOT.jl/pull/8#issuecomment-606596770>
>
> Here is a root script to produce the tree:
> \`\`\`c++
> \#include "TString.h"
> \#inclu…de "TFile.h"
> \#include "TTree.h"
> 
> \#include \<string\>
> \#include "stdlib.h"
> 
> 
> int main() {
> 
> TFile \*f = new TFile("test\_tree.root", "recreate");
> TTree \*t = new TTree("mytree", "mytree");
> 
> auto app = "XYZE";
> 
> const uint Np = 100;
> //
> double v\[Np\]\[4\];
> for (uint n=0; n\<Np; n++)
> for (uint i=0; i\<4; i++)
> t-\>Branch(TString::Format("p%d\_%c",n, app\[i\]), &v\[n\]\[i\]);
> 
> const int Nev = 10000;
> 
> for (uint i=0; i \<= Nev; i++) {
> 
> for (uint n=0; n\<Np; n++)
> for (uint i=0; i\<4; i++)
> v\[n\]\[i\] = (rand() % 100) / 100.0;
> //
> t-\>Fill();
> }
> 
> t-\>Write();
> f-\>Close();
> 
> return 0;
> }
> \`\`\`
> 
> then, you can try (only \`\*\_X\` branches)
> \`\`\`julia
> @time let
> f = TFile(joinpath("test\_tree.root"))
> t = f\["mytree"\]
> t.p1\_X + t.p2\_X
> end
> \`\`\`
> will take forever (11 sec.).
> 
> Surely faster to convert the whole tree and, then, access.
> \`\`\`
> @btime let
> f = TFile(joinpath("test\_tree.root"))
> t = f\["mytree"\]\[1:end\]
> t.p1\_X + t.p2\_X
> end # 351.006 ms
> \`\`\`
> 
> Creating the three with only needed branches is the fastest option
> \`\`\`julia
> @btime let
> f = TFile(joinpath("test\_tree.root"))
> t = f\["mytree"\]
> tT = getindex(t, eachindex(t), \["p1\_X", "p2\_X"\])
> #
> tT.p1\_X + tT.p2\_X
> end # 328.482 ms
> \`\`\`
> The difference between the last two scales with the number of branches.

The file has 300\_000 rows, the size of 10MB

Some more retails are below

the `Particle` is a simple mutable struct, the `process` function creates a named tuple of for particles

```julia
using UpROOT
using Parameters
#
@with_kw mutable struct Particle
    E::Float64
    px::Float64
    py::Float64
    pz::Float64
end
#
process(row) = 
   (pb=Particle(
        px = row.pBeamX,
        py = row.pBeamY,
        pz = row.pBeamZ,
         E = row.pBeamE),
    pr = Particle(
        px = row.pRecoilX,
        py = row.pRecoilY,
        pz = row.pRecoilZ,
         E = row.pRecoilE),
    pπ = Particle(
        px = row.pPimX,
        py = row.pPimY,
        pz = row.pPimZ,
         E = row.pPimE),
    pη = Particle(
        px = row.pEtaX,
        py = row.pEtaY,
        pz = row.pEtaZ,
         E = row.pEtaE))

```

---

<div class="post-metadata">

### Author: ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)
#### Post date: [August 11, 2020, 9:18am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/2 "2020-08-11T09:18:15Z")

</div>

Pin @oschulz. Any ideas?  
Thanks

---

<div class="post-metadata">

### Author: ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)
#### Post date: [August 11, 2020, 9:28am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/3 "2020-08-11T09:28:23Z")

</div>

Just found a typo in the second execution:  
it should be `process.(tr)` instead of `process.(t)`. Then it takes 1s

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/c/cc83bf04615ff6dfc9e60254572b22f218bbfba7.png)

What remains unclear why processing the tree directly is so much slower. It contains  
the only branches that I use

![image](https://global.discourse-cdn.com/julialang/original/3X/2/d/2dccc30a9f4c04793c216936578ae852a3bebe34.png)

---

<div class="post-metadata">

### Author: ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)
#### Post date: [August 11, 2020, 9:34am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/4 "2020-08-11T09:34:46Z")

</div>

Well, here is the shortest representation of the problem:  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/c/1c633a3140c988a4bc544f1023a841bbe3262df4.png)  
A few orders of magnitude difference. Interesting.

---

<div class="post-metadata">

### Author: ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)
#### Post date: [August 11, 2020, 10:09am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/5 "2020-08-11T10:09:28Z")

</div>

Conversion of the TTree (in disk) to Table (in memory) is done when `getindex` is called.  
It seems that the conversion is happening for every index when broadcast

---

<div class="post-metadata">

### Author: ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)
#### Post date: [August 11, 2020, 10:26am UTC](https://discourse.julialang.org/t/tables-jl-uproot-jl-and-the-speed/44737/6 "2020-08-11T10:26:54Z")

</div>

Probably best if we discuss this on your GibHub issue ([https://github.com/JuliaHEP/UpROOT.jl/issues/12](https://github.com/JuliaHEP/UpROOT.jl/issues/12)), in case we need to make changes to UpROOT.jl
