Help with UpROOT in Julia

rushabhgala · March 4, 2024, 7:22pm

This is my code

using UpROOT
f = TFile("test.root")
println(keys(f))

which gives the output
["raw;1", "mul;1", "lar;1", "mul_lar;1"]

There are about a 100 histograms in raw, mul, lar, mul_lar of type TH1F
How do I plot them?
For example there is a TH1F named all in raw so when I try to do
all = f["raw/all"]
I get this error

┌ Warning: Conversion of python type Undefined_TH1F to a Julia type not supported
└ @ UpROOT ~/.julia/packages/UpROOT/Onjve/src/pyjlconv.jl:62
PyObject <Undefined_TH1F at 0x0002b05726d0>

I’ve not used python much but I remember that TH1F can be saved as a numpy object or a dict or something else, what is the equivalent in julia?
The documentation for this julia wrapper isn’t helping me either
https://juliahep.github.io/UpROOT.jl/stable/

P.S. This is my first time using julia so apologies if I am overlooking something obvious

carstenbauer · March 4, 2024, 8:29pm

I’m not from HEP but have you considered the pure-Julia reader UnROOT.jl or is there a particular reason you want to use UpROOT.jl (which wraps the same named Python package)? (cc @jling who might know more here)

rushabhgala · March 4, 2024, 8:44pm

I did consider and use UnROOT.jl but it isn’t able to read TH1F either

It would even work for me if I can use PyCall to use the python version of UpROOT to read those histograms and save it to a julia array(?) or something similar
I followed this (Getting started guide — Uproot documentation) uproot tutorial for python in julia using PyCall but I got the same not defined/not found error

julia> f.classnames()
Dict{Any, Any} with 4 entries:
  "hpxpy;1"  => "TH2F"
  "hpx;1"    => "TH1F"
  "hprof;1"  => "TProfile"
  "ntuple;1" => "TNtuple"

julia> f["hpx"].all_members
ERROR: KeyError: key "hpx" not found
Stacktrace:
 [1] __getproperty
   @ ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:313 [inlined]
 [2] getproperty
   @ ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:317 [inlined]
 [3] getindex(o::PyObject, s::String)
   @ PyCall ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:355
 [4] top-level scope
   @ REPL[39]:1

julia> f["hpx"].member["fName"]
ERROR: KeyError: key "hpx" not found
Stacktrace:
 [1] __getproperty
   @ ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:313 [inlined]
 [2] getproperty
   @ ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:317 [inlined]
 [3] getindex(o::PyObject, s::String)
   @ PyCall ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:355
 [4] top-level scope
   @ REPL[40]:1

jling · March 4, 2024, 9:49pm

in the meanwhile you can manually use PythonCall + uproot to get it if you really want

julia> using PythonCall

julia> up = pyimport("uproot")
Python: <module 'uproot' from '/tmp/jl_nrWpqL/.CondaPkg/env/lib/python3.12/site-packages/uproot/__init__.py'>

julia> up.open("/tmp/hepdata-example.root")["hpx"]
Python: <TH1F (version 1) at 0x7c597ddc7a40>

julia> up.open("/tmp/hepdata-example.root")["hpx"].to_numpy()
Python:
(array([2.000e+00, 3.000e+00, 1.000e+00, 1.000e+00, 2.000e+00, 4.000e+00,
       6.000e+00, 1.200e+01, 8.000e+00, 9.000e+00, 1.500e+01, 1.500e+01,
       3.100e+01, 3.500e+01, 4.000e+01, 6.400e+01, 6.400e+01, 8.100e+01,
       1.080e+02, 1.240e+02, 1.560e+02, 1.650e+02, 2.090e+02, 2.620e+02,
       2.970e+02, 3.920e+02, 4.320e+02, 4.660e+02, 5.210e+02, 6.040e+02,
       6.570e+02, 7.880e+02, 9.030e+02, 1.079e+03, 1.135e+03, 1.160e+03,
       1.383e+03, 1.458e+03, 1.612e+03, 1.770e+03, 1.868e+03, 1.861e+03,
       1.946e+03, 2.114e+03, 2.175e+03, 2.207e+03, 2.273e+03, 2.276e+03,
       2.329e+03, 2.325e+03, 2.381e+03, 2.417e+03, 2.364e+03, 2.284e+03,
       2.188e+03, 2.164e+03, 2.130e+03, 1.940e+03, 1.859e+03, 1.763e+03,
       1.700e+03, 1.611e+03, 1.459e+03, 1.390e+03, 1.237e+03, 1.083e+03,
       1.046e+03, 8.880e+02, 7.520e+02, 7.420e+02, 6.730e+02, 5.550e+02,
       5.330e+02, 3.660e+02, 3.780e+02, 2.720e+02, 2.560e+02, 2.000e+02,
       1.740e+02, 1.320e+02, 1.180e+02, 1.000e+02, 8.900e+01, 8.600e+01,
       3.900e+01, 3.700e+01, 2.500e+01, 2.300e+01, 2.000e+01, 1.600e+01,
       1.400e+01, 9.000e+00, 1.300e+01, 8.000e+00, 2.000e+00, 2.000e+00,
       6.000e+00, 1.000e+00, 0.000e+00, 1.000e+00], dtype=float32), array([-4.  , -3.92, -3.84, -3.76, -3.68, -3.6 , -3.52, -3.44, -3.36,
       -3.28, -3.2 , -3.12, -3.04, -2.96, -2.88, -2.8 , -2.72, -2.64,
       -2.56, -2.48, -2.4 , -2.32, -2.24, -2.16, -2.08, -2.  , -1.92,
       -1.84, -1.76, -1.68, -1.6 , -1.52, -1.44, -1.36, -1.28, -1.2 ,
       -1.12, -1.04, -0.96, -0.88, -0.8 , -0.72, -0.64, -0.56, -0.48,
       -0.4 , -0.32, -0.24, -0.16, -0.08,  0.  ,  0.08,  0.16,  0.24,
        0.32,  0.4 ,  0.48,  0.56,  0.64,  0.72,  0.8 ,  0.88,  0.96,
        1.04,  1.12,  1.2 ,  1.28,  1.36,  1.44,  1.52,  1.6 ,  1.68,
        1.76,  1.84,  1.92,  2.  ,  2.08,  2.16,  2.24,  2.32,  2.4 ,
        2.48,  2.56,  2.64,  2.72,  2.8 ,  2.88,  2.96,  3.04,  3.12,
        3.2 ,  3.28,  3.36,  3.44,  3.52,  3.6 ,  3.68,  3.76,  3.84,
        3.92,  4.  ]))

rushabhgala · March 4, 2024, 11:25pm

Is PyCall different from PythonCall?

I am unable to get the array.

Here is what I get

julia> using PyCall

julia> up = pyimport("uproot")
PyObject <module 'uproot' from '/Users/rushabh/miniforge3/envs/myenv/lib/python3.9/site-packages/uproot/__init__.py'>

julia> up.open("/Users/rushabh/legend-base/LFS/hepdata-example.root")["hpx"]
ERROR: KeyError: key "hpx" not found
Stacktrace:
 [1] __getproperty
   @ ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:313 [inlined]
 [2] getproperty
   @ ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:317 [inlined]
 [3] getindex(o::PyObject, s::String)
   @ PyCall ~/.julia/packages/PyCall/1gn3u/src/PyCall.jl:355
 [4] top-level scope
   @ REPL[4]:1

Edit : I am trying to install PythonCall but facing errors

error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ pythoncall *  does not exist (perhaps a typo or a missing channel).
critical libmamba Could not solve for environment specs
ERROR: failed process: Process(`/Users/rushabh/.julia/artifacts/5767a45b62528e0be4c40dbae502ad3e132f1d62/bin/micromamba -r /Users/rushabh/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root create -y -p /Users/rushabh/.julia/environments/v1.9/.CondaPkg/env --override-channels --no-channel-priority "python[version='>=3.7,<4',channel='conda-forge',build='*cpython*']" "pythoncall[version='*']" -c conda-forge`, ProcessExited(1)) [1]

jling · March 5, 2024, 2:51am

yeah, if you’re using PyCall, you need to manually call the getitem

julia> using PyCall

julia> up = pyimport("uproot")
PyObject <module 'uproot' from '/home/akako/.local/lib/python3.10/site-packages/uproot/__init__.py'>

julia> up.open("/tmp/hepdata-example.root").__getitem__("hpx").to_numpy()
(Float32[2.0, 3.0, 1.0, 1.0, 2.0, 4.0, 6.0, 12.0, 8.0, 9.0  …  14.0, 9.0, 13.0, 8.0, 2.0, 2.0, 6.0, 1.0, 0.0, 1.0], [-4.0, -3.92, -3.84, -3.76, -3.68, -3.6, -3.52, -3.44, -3.36, -3.2800000000000002  …  3.2800000000000002, 3.3600000000000003, 3.4400000000000004, 3.5200000000000005, 3.6000000000000005, 3.6799999999999997, 3.76, 3.84, 3.92, 4.0])

rushabhgala · March 5, 2024, 4:05pm

Thank you so much! Also I just realized you were the one who was on the github issue page as well. Thank you for helping me on both fronts.
I do have another question as I am using julia since only a couple of days. This is what my TH1F output looks like for the actual file I want to use

c = up.open("test.root").__getitem__("raw/all").to_numpy()
(Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0  …  7991.0, 7992.0, 7993.0, 7994.0, 7995.0, 7996.0, 7997.0, 7998.0, 7999.0, 8000.0])

The first set of [ ] has bin values and the second set of [ ] has bin numbers from 0-8000 I’m gonna have 100s of these TH1Fs so What’s the best way to store them in julia? As an array? something else?
Thank you

jling · March 5, 2024, 5:23pm

you can reconstitute these data to a FHist histogram:

julia> using FHist: Hist1D

julia> counts, edges = up.open("/tmp/hepdata-example.root").__getitem__("hpx").to_numpy()
(Float32[2.0, 3.0, 1.0, 1.0, 2.0, 4.0, 6.0, 12.0, 8.0, 9.0  …  14.0, 9.0, 13.0, 8.0, 2.0, 2.0, 6.0, 1.0, 0.0, 1.0], [-4.0, -3.92, -3.84, -3.76, -3.68, -3.6, -3.52, -3.44, -3.36, -3.2800000000000002  …  3.2800000000000002, 3.3600000000000003, 3.4400000000000004, 3.5200000000000005, 3.6000000000000005, 3.6799999999999997, 3.76, 3.84, 3.92, 4.0])

julia> FHist.Hist1D(FHist.Histogram(edges, counts))
edges: [-4.0, -3.92, -3.84, -3.76, -3.68, -3.6, -3.52, -3.44, -3.36, -3.2800000000000002  …  3.2800000000000002, 3.3600000000000003, 3.4400000000000004, 3.5200000000000005, 3.6000000000000005, 3.6799999999999997, 3.76, 3.84, 3.92, 4.0]
bin counts: Float32[2.0, 3.0, 1.0, 1.0, 2.0, 4.0, 6.0, 12.0, 8.0, 9.0  …  14.0, 9.0, 13.0, 8.0, 2.0, 2.0, 6.0, 1.0, 0.0, 1.0]
total count: 74994.0

see some plotting examples: Makie Plotting · FHist.jl

We’re currently trying to re-design the API for FHist.jl to make it more intuitive: Main API change for v0.11 🎂 by Moelf · Pull Request #95 · Moelf/FHist.jl · GitHub