Broadcasting error diffeqflux

Specific Domains Machine Learning
4

1, Yeah, that’s why I transposed the X_train and y_train

I saw that too by now :slight_smile:

To which tutorial are you referring? I know of https://diffeqflux.sciml.ai/stable/examples/augmented_neural_ode/ where the model is constructed like

diffeqarray_to_array(x) = reshape(gpu(x), size(x)[1:2])

function construct_model(out_dim, input_dim, hidden_dim, augment_dim)
    input_dim = input_dim + augment_dim
    node = NeuralODE(Chain(Dense(input_dim, hidden_dim, relu),
                           Dense(hidden_dim, hidden_dim, relu),
                           Dense(hidden_dim, input_dim)) |> gpu,
                     (0.f0, 1.f0), Tsit5(), save_everystep = false,
                     reltol = 1e-3, abstol = 1e-3, save_start = false) |> gpu
    node = augment_dim == 0 ? node : (AugmentedNDELayer(node, augment_dim) |> gpu)
    return Chain((x, p=node.p) -> node(x, p),
                 diffeqarray_to_array,
                 Dense(input_dim, out_dim) |> gpu), node.p |> gpu
end

You can see that the output dimension of node is equal to its input dimension. This has to be the case, because it solves an ODE (\mathbf{h(t)}\in R^d).
\mathbf{h}'(t) = \mathbf{f}(\mathbf{h(t))} where \mathbf{f} is a map R^d\to R^d. f is nn and h(0) is the input X_train.
I don’t think I can make a qualified comment on the internal structure of the network.

The reduction to the dimension of y is done by the second network, but not by the NODE.

Also, the line I marked as ‘superfluous’ is not present.

1 Like