Flux for deep learning in particle physics

Specific Domains Machine Learning
1

There is a CodaLab code competition for applying deep learning to particle physics
https://competitions.codalab.org/competitions/19818#learn_the_details

As I have a background in the field (see me? See a CERN PhD which used FORTRAN?) I am toying with the idea of using this as a Flux learning project.
The question I have is - where should I install and run Flux?
Is Flux on Juliabox? I dont mind paying for a GPU instance if that is what is needed.
Or should I start with an AWS or Google instance?

Second stupid question - it is ridiculously easy to copy a dataset into Juliabox? I guess it must be or it would be essentially useless!

2 Likes
2

How Big is the dataset?

3

I’d personally suggest an AWS or Google cloud instance. For now AWS has slightly easier access to GPUs (Google requires manually requesting GPU quota first) while Google has a slightly easier user interface (the Cloud Shell is an amazing idea), slightly cheaper GPU pricing, and a nice free trial deal.

4

I have used Mocha and MXNet on EC2 GPU instances, it was pretty straightforward. I suppose with Flux, it would be similarly easy.

5

Thankyou all. Regarding the dataset the training dataset is an HDF5 file with 6K events.
Each event is a 192x192 array. I Cannot actually find the file on GitHub to report the size!

6

So something like 1,8 gigabytes? I’m not sure that’ll fit on your juliabox account. It may be possible to allocate more disk space by request though, if you use it it to make a cool Flux example :slight_smile:

7

Just click on remix here:

And you get a complete IJulia like environment with a tesla k80, Flux + CuArrays readily installed :wink:

Have fun playing around with it! Signup code: juliacon

Flux + CuArrays is not 100% stable right now, but I will try to keep it up to date whenever there is an improvement.

Best,
Simon

2 Likes