Broadcasting over a Flux.DataLoader

Specific Domains Machine Learning
3

Thanks very much for your reply.

I am still very puzzled after reading the docs again. I’m working on a simplified example but I cannot make it work. Maybe you can help me get the simple case working and then I can figure out my actual use case?

# Suppose I have four sentences, each 20 words long, 
# w/ embedding dim per word of 10

a1 = rand(Float32, 10,20)
a2 = rand(Float32, 10,20)
a3 = rand(Float32, 10,20)
a4 = rand(Float32, 10,20)
fake_train = cat(a1, a2, a3, a4; dims = 3)


fake_labels = Flux.onehotbatch([0,1,0,1], [0,1])
# First question: is this step necessary to give the labels a similar shape to fake_train??
fake_labels = reshape(fake_labels, (2,1,4)) 

DL = Flux.DataLoader((data = fake_train, label=fake_labels), batchsize=2)


l1 = LSTM(10, 5)
d1 = Dense(5,2)
function model(x, scanner, encoder)
    state = scanner(x)[:,end]
    reset!(scanner)
    encoder(state)
end 


ps = params(l1, d1)
opt = ADAM(1e-3)

loss(x,y)= Flux.logitbinarycrossentropy(model(x, l1, d1), y)
# returns a number, not an error, so the loss function is working, I believe
loss(fake_train[:,:,1], fake_labels[:,:,1]) 

for i = 1:3
    @info i 
    Flux.train!(loss, ps, DL, opt)
end 

ERROR: LoadError: MethodError: no method matching loss(::NamedTuple{(:data, :label), Tuple{Array{Float32, 3}, Array{Bool, 3}}})
# this error makes sense given the way I defined loss(x,y)

The docs say “If d is a tuple of arguments to loss call loss(d...) , else call loss(d) .” I tried to follow advice here and do literally that (i.e., train!(x -> loss(x...), ...), but that also generated an error:
ERROR: LoadError: BoundsError: attempt to access 5×20×2 Array{Float32, 3} at index [1:5, 20]

I have also tried defining loss(x,y) = logitbinarycrossentropy.(... (i.e, broadcasting) or loss((x,y)) = ... (i.e., defining the loss over tuples directly), but these generate errors of their own.

Maybe I am a little closer to an MWE now…?