High Energy Physics (HEP) folks on Julia?

question

#1

Maybe it’s not the most suitable domain but imo close enough.
For reference, ROOT refers to ROOT/CERN


Really enjoyed using Julia for a while and trying to apply it to more study/research replated works.
I noticed there is a HEPJulia/ROOT.jl which is essentially a wrapper of its original C++ library, but now that it’s undermaintained & Cxx is broken in 0.7/1.0, I’m wondering if any of you has been working privately on such a project because based on the attempts given I found it is very hard for people other than those came up with the binary encoding decades ago to parse their stuff.

The similar projects in Python/JavaScript would be uproot and jsroot which basically has their own full-blown and native (and very complicated) binary parser per the specification of binary files.

It would also be great if anyone has experience/suggestion on how to process binaries un-painfully.


#2

Pinging @oschulz :slight_smile:


#4

Thanks for the ping, @giordano!

@jerryling315, we are using ROOT.jl in production on Julia v0.6 to read and write ROOT files (with custom streamers, so something like uproot wouldn’t work). So ROOT.jl is definitely not abandoned, I’m just waiting for Cxx.jl to port it to Julia v1.0 - Keno told me that Cxx.jl should be available on v1.0 soon.

(See also https://github.com/JuliaHEP/ROOT.jl/issues/15).


#5

Thanks for replying!

Though, do you think it’s possible that we can make something like uproot someday?


#6

Hey there, and welcome!

I’m using ROOT as little as possible, but if you’re looking for a way to wrap C++ dependencies, I can recommned CxxWrap. Used it for the LCIO file format, and currently working on a fastjet wrapper.


#7

How about Geant4? I noticed @oschulz has an abandoned? Geant4.jl repo. Is somebody using Geant4 from julia? I tried to use Geant4 myself with Cxx.jl and also Clang.jl and could run a very basic hello world simulation a year or two ago. But I had lots of problems with gui and multithreading and gave up.


#8

My work depends on both ROOT and Geant4, but honestly I try to avoid both of them when possible :wink:

Unfortunately our data format is based on ROOT with custom classes but that’s basically it, I personally wouldn’t care about interfacing all the algorithms and other classes in ROOT, I’m only interested in the data format.
It would be nice if we had something similar as uproot which I am currently utilising to access the data from our detector and simulations without installing ROOT and another the huge project specific framework (doesn’t build on macOS or GCC newer than 4.8.x and it’s almost impossible to interface it via Cxx.jl or CxxWrap since everything is template based).

Our current solution is converting the ROOT files to HDF5 but direct access would definitely be much more appreciated.

Regarding Geant, well… my hope is having a particle interaction simulator written in Julia.

Maybe we should team up and cook a stone soup…?


#9

This is indeed my feeling as well. At the end of the day most production/paper result need to be done in ROOT but most of the time are spent on analyzing simple histogram/plots inside .root files which I don’t really need to call ROOT built-in functions (thousands of them) because with the extracted data Julia/Python often can be way easier to manipulate.


#10

I don’t think that ROOT is necessary for an official publication. I’d rather say that it has to be well documented, reproducible incl. correct data provenance and that’s it.

This can be done in any language…


#11

How about Geant4? I noticed @oschulz has an abandoned? Geant4.jl repo.

It’s currently still a bit of an experiment - but it’s not abandoned, either. I am able to run a simple Geant-4 simulation from Julia with it (using embedded C++ code to specify the geometry, etc., try the “dev” branch), but I ran into trouble (library loading) when I tried to open a GL visualization window - will probably need LD_PRELOAD or so. I do have plans to continue working on it, as soon as @Keno has ported Cxx.jl to Julia v1.0, though ROOT.jl and ROOTFramework.jl will have priority.


#12

One think I’ve been thinking about is using JSROOT for plotting. It would enable high-quality interactive plots in Jupyter notebooks, etc. without depending on ROOT itself (which is quite a heavy dependency). But I never found time to actually work on it. I would be a bit more tricky with JupyterLab now - from what I understand it’s not as easy to use custom JavaScript with JupyterLab, needs some extra steps, but I’m not an expert.


#13

I do have a ROOT wrapper (not registered yet) called ROOTFramework.jl (depends on ROOT.jl) that adds some Julian wrappers around ROOT. It allow allows you to use the ROOT GUI from within a Julia session (see examples directory), including TBrowser and fit panels. Currently runs on Julia v0.6 only, of course, but I definitely will port it to v1.0 as soon as Cxx.jl (and ROOT.jl) are ported. With ROOT.jl and ROOTFramework.jl you can make ROOT plots from within Julia, and open plot windows. Not in Jupyter/JupyterLab though, would need integration of ROOTJS (see my other post).

In principle, we’re now trying to avoid ROOT in our group, where possible - it is an awkward dependency. But we can’t always avoid ROOT (interfaces to other groups, etc.), so good ROOT integration in Julia is important to me. It’s also an important bridge for people who want to slowly (or just partially) migrate from ROOT to pure Julia solutions. Anyone interested to join in is very welcome, of course!


#14

@tamasgal: My work depends on both ROOT and Geant4, but honestly I try to avoid both of them when possible

We’re in a similar position. :slight_smile: Though Geant4 is actually unavoidable for us, so at some point, I’d like to have a good way to run it from inside of Julia (my Geant4.jl is just a first step).

Unfortunately our data format is based on ROOT with custom classes … It would be nice if we had something similar as uproot

Same for us, though some of the files we have to read don’t just use custom classes but also custom streamers, so there’s not way of using ROOT (at least behind the scences) to read them.

it’s almost impossible to interface it via Cxx.jl or CxxWrap since everything is template based).

Templates shouldn’t be a problem for Cxx.jl.

Our current solution is converting the ROOT files to HDF5 but direct access would definitely be much more appreciated.

We do both, actually - if we need to read files multiple times, conversion to HDF5 gives us better performance. So we often read from ROOT (via ROOT.jl), apply some fixed data pre-processing that we need anyhow, and then write more-or-less temporary HDF5 files.

We’re trying to convince people to use HDF5 as the primary storage format in the future, though … we’ll see. The Python users in other groups aren’t too happy with ROOT as a storage format, either.


#15

For me its the same thing. I only need ROOT for IO, I would prefer julia algorithms + plots. Having a particle monte carlo code in pure julia would be amazing. If we had this, I guess I would never touch Geant4+ROOT again (I actually like Geant4, I just like julia much more :smile: ).
Its definitely a project I would contribute to. From time to time I toyed with the idea of trying to do this. The main obstacle for me was, that I could not identify a tiny subset that

  1. is somewhat useful
  2. can be implemented with reasonable effort

My applications are in medical physics. For me the smallest subset that is useful would be:

  • Geometry: Just slabs stacked on top of each other. Or maybe something like a CT image. Both should be easy.
  • Physics: Photons, electrons, positrons say 100keV-10MeV. Photons are easy, charged particles are harder.

So compared to geant4 this is tiny, but I getting electrons right seems already quite involved to me.


#16

You could always call uproot via PyCall, though I agree that in the long run a native package would be nicer.


#17

Maybe this is a good starting point for a julia implementation: https://github.com/go-hep/hep/tree/master/rootio


#18

While I still need to use the actual ROOT IO implementation in some cases (files with custom streamers), in many cases people just deal with ROOT files that use standard types.

A wrapper or native clone of uproot in Julia would be nice - uproot has the advantage that it actually utilizes the internal columnar storage of ROOT files - ironically something that the official C++ ROOT itself doesn’t let you do.