Flattened data vs Flux.flatten layer

Specific Domains Machine Learning
1

Hi all,

I am trying to use the following MLP model (for MNIST) in the model zoo as a template for my work, and I have this problem:
https://github.com/FluxML/model-zoo/blob/master/vision/mlp_mnist/mlp_mnist.jl

I remove the flattening data (lines 18 - 19) and try to add a Flux.flatten layer before the Dense layer, like this:

return Chain(
              Flux.flatten,
              Dense(prod(imgsize), 32, relu),
              Dense(32, nclasses))

but I got very bad training/testing accuracy (it still runs though). I thought they should be the same. Any idea?
Thanks.

2

I think they should be too. Can you confirm the shape of x is correct before it’s fed into the network (i.e. after https://github.com/FluxML/model-zoo/blob/master/vision/mlp_mnist/mlp_mnist.jl#L84)? If everything looks right there (the batch is 28 x 28 x [batch size]) and you experience this poor performance across multiple runs, please do file an issue :slight_smile:

3

I check the shape of x before its fed into the networks and it is of the shape (28, 28, 256), which looks correct. I just submit the issue:
https://github.com/FluxML/model-zoo/issues/316