In Flux, the batch size is last. So, for image data, it follows WHCN format. Also, you only call the model on a batch at a time. So, d(x) assume that x is a batch of data. If you want one batch of five samples and 64 features, then you need
d = Dense(64, 2)
# apply to a single batch of size 5 w/ 64 features
d(rand(Float32, 64, 5))
# create a vector of 10 batches
# each batch is size 5 w/ 64 features
xs = [rand(Float32, 64, 5) for _ in 1:10]
# loop over the batches and apply the model
ys = []
for x in xs
y = d(x)
push!(ys, y)
end
# use map instead of a loop
ys = map(d, xs)
# use broadcasting
ys = d.(x)
Also, maybe you already have data that you need to split into batches. You can use the following:
using Iterators: partition
# create a 100 samples w/ 64 features each
X = rand(Float32, 64, 100)
# split into batches of size 5
xs = [X[:, i] for i in partition(1:100, 5)]
Lastly, the methods above are using basic Julia functionality which is usually sufficient for simple arrays of data like X. But for complex datasets, you might want to use data loaders with built-in batching like Flux.DataLoader or DataLoaders.DataLoader.
Basically, what I need is to overload this function with this new version (taken from version 0.12):
function (a::Dense)(x::AbstractArray)
W, b, σ = a.W, a.b, a.σ
σ.(W*x .+ b)
# reshape to handle dims > 1 as batch dimensions
sz = size(x)
x = reshape(x, sz[1], :)
x = σ.(W*x .+ b)
return reshape(x, :, sz[2:end]...)
end
How can I do it? In my tests, Julia still prefers the Flux function, not mine
You need to either add import Flux: Dense at the start of your code. Or do (a::Flux.Dense)(x::AbstractArray) in your definition. (you need to override/extend the function in the Flux namespace).
Also, adding that override would make your code run, but I don’t think it will do want you want it to. Maybe I just misunderstood your explanation from before though.