Using Neural Network for regression (in Julia)


#1

Hello,

I’d like to use Julia for a small ML project. My dataset is about 2000 samples and 142 features.

I was hoping to use a simple Neural Network to do this task (with a ReLU neuron as the output node?). However, I just don’t know which package I should use for that! There are a lot of available package, but they seem to be aimed at complex problems/solutions.

Thanks for any suggestions and help! Bear in mind that I’m a total novice in this field. :slight_smile: Hence, I’m open to alternatives, but I’m looking for a fairly simple approach to begin with.


#2

To my knowledge, deep learning is state-of-the-art on problems where the spatial/time structure can be exploited (via convolution/recursion), but it hasn’t made great strides elsewhere. With 2000 samples X 142 features, I would consider other classical (regularized) ML algorithms, like random forests or gaussian processes; they are much simpler to setup. Do you have a specific reason to favour deep learning?


#3

TensorFlow.jl works fine on simple problems. You can see a simple example of logistic regression in its README.


#4

No deep reason to use deep learning. I was more looking at a simpler NN. I just thought that having so many features that a NN would be well suited for modelizing the non-linear relationships.

The use case is I want to predict the number of ticket sold at a ski stations, based on these features.


#5

Thanks, I’ll take a look at it!

However, I dont need logistic regression as far as I understand.


#6

Logistic regression is the simplest example of a neural network model, so I just linked to it for demonstration purposes. TensorFlow is for general deep learning.


#7

Ok! I see what you meant now. Thanks for the added details!


#8

I might be biased, but that sounds like a typical use case for scikit-learn-like modelling, whether directly through PyCall, or through ScikitLearn.jl. See eg. the Decision Tree example (could also use a random forest with nearly the same code), or the Gaussian processes example. PCA might help, too.

If you try deep learning, watch out for overfitting.


#9

Thanks!

Somehow I forgot the existence of ScikitLearn.jl. This is looking like the package I need for this small project.

edit - Any reason why this package is not listed in JuliaML org?


#10

For a list of all ML projects, Julia.jl is a good ressource.


#11

A regression example using MXNet.jl is here:
https://github.com/dmlc/MXNet.jl/blob/master/examples/regression-example.jl


#12

Thanks for pointing that out. I’ll look at it more closely.

Yet, I don’t see ScikitLearn.jl there :slight_smile: