I just got a simple classification example working with
Flux.jl. I was originally looking for something like it, but did not find anything. I have tried to make comments along the way on the level I would have needed yesterday The comments are sort of in the way of the code, but they are also placed exactly where most needed, which I think helps newcomers (like me) understand what is going on.
I think that the example and all the comments can be overwhelming, but that everything is explained so that the reader always has the option to understand what goes on. I was missing that in the examples that I found.
I am looking to hear - would this example be welcome in the flux Model Zoo? And if yes, are there suggestions for improvement?
One improvement is adding titles to the plots, and automating the iterative training+plotting. I was unable to make this happen in my first attempts.
##! Classifying input points as above or below a given line #* Comments are maked by a hashtag and star, which looks good with the "Better Comments" extension for VSCode. #* Code blocks are seperated by a double-hashtag. #* Run the script line by line, or block by block. using Flux using Flux: train!, params, onehotbatch, onecold #* The line used as reference for above or below y(x) = 4x + 2 #* Function to give a vector of output classifiction based on an input matrix. #* A full column is an input. Row 1 is x-coordinate, row 2 is y-coordinate function classify(points) outputs =  for col in eachcol(points) y(col) < col && push!(outputs, :above) y(col) ≥ col && push!(outputs, :below) end return outputs .|> identity #* identity constrains the element-type, which makes Makie.jl happy end #* Generating the data. x denotes inputs into the #* ArtificialNeuralNetwork (ANN), y denotes outputs (classification) inputs_train = vcat(0:0.01:1 |> transpose, [y(x) + randn() * x for x = 0:0.01:1] |> transpose) inputs_test = vcat(0.005:0.01:1 |> transpose, [y(x) + randn() * x for x = 0.005:0.01:1] |> transpose) inputs_train |> classify #* Checking that we get outputs that make sense ## #* Using the `classify` function to generate the correct classification for this supervised learning outputs_train = classify(inputs_train) outputs_test = classify(inputs_test) #* onehot encoding to translate the classifications to something that makes more sense to the ANN categories = [:above, :below] outputs_train_encoded = onehotbatch(outputs_train, categories) outputs_test_encoded = onehotbatch(outputs_test, categories) ##* Plotting the data generated thus far, mostly as a sanity-check #* AlgebraOfGraphics provides convenient color by category using AlgebraOfGraphics #* Because `data` will be used for something else: using AlgebraOfGraphics: data as AOGdata axis = (width = 500, height = 500) #* The axis we will plot into #* Putting our data into a named tuple of vectors, which is a DataFrame as far as AlgebraOfGraphics is concerned. #* Actually making a dataframe has the same result. train_data = ( xs = inputs_train[1, :], ys = inputs_train[2, :], label = outputs_train ) #* Let's see if our dataset looks right, by plotting it. #* If you don't understand the plotting below, but want to, see the AlgebraOfGraphics documentation. Not important for Machine Learning train_data_plot = AOGdata(train_data) * mapping(:xs, :ys, color = :label) the_line = AOGdata((xs = 0:0.1:1, ys = y.(0:0.1:1))) * mapping(:xs, :ys) * linear() draw(train_data_plot + the_line; axis) ## #* Putting the data in a tuple inside a vector, because it is how Flux likes it's data. data = [(inputs_train, outputs_train_encoded)] #* Making the ANN: model = Chain( Dense(2, 5), #* 2 inputs because an input point has 2 coordinates Dense(5, 5, relu), #* Arbitraty choice Dense(5, 2, σ) #* 2 outputs because there are 2 categories. σ activation so that output can be interpreted as probabilities. ) #* Defining the loss-function. loss(inputs, outputs) = Flux.Losses.logitbinarycrossentropy(model(inputs), outputs) #* crossentropy is good for classification, binary because there are only 2 categories, and logit because of numerical stability (as reccomended in the docstring for `crossentropy`) opt = ADAM() #* ADAM works for classification problems parameters = params(model) #* A variable that referres to the model parameters. ## #* See performance without training decisions = onecold(model(inputs_test), categories) model_test_data = ( xs = inputs_test[1, :], ys = inputs_test[2, :], decisions = decisions ) model_test_plot = AOGdata(model_test_data) * mapping(:xs, :ys, color = :decisions) + the_line draw(model_test_plot; axis) ## #* Train for 500 rounds, plotting the result in the end of this block. Run this block several times to watch the network learn! for _ = 1:500 train!(loss, parameters, data, opt) end #* Onecold does the reverse of onehot, making translating the categoies from computer to human #* The output is the most likely category decisions = onecold(model(inputs_test), categories) count(==(1), decisions .== outputs_test) #* Count how many the ANN got right, if wanted #* Plotting the test-data (which the model has not trained on), colored by the model's categorization. model_test_data = ( xs = inputs_test[1, :], ys = inputs_test[2, :], decisions = decisions ) model_test_plot = AOGdata(model_test_data) * mapping(:xs, :ys, color = :decisions) + the_line draw(model_test_plot; axis)
Sany check of model data: