Rewriting "Example 2: Calling Julia from Python" in Julia

General Usage
,
1

I try to take the script shown in the link to Julia.
example-2-calling-julia-from-python
But in the following lines:

loss = jl.seval("m -> (x, y) -> Flux.Losses.me(m(x), y)")(model)

jl.Flux.train_b(
   loss,
   jl.Flux.params(model),
   jl.Vector[jl.Tuple]([batch(100) for _ in range(2000) ]),
   jl.ADAM (0.01) ,
)

I don’t know how to write the code in Julia

This is all I have achieved:

using Plots
pyplot()
using Random

Random.seed!(1234)

function batch(n)
    x = sort(rand(-1.1:0.01:1.1, n))
    y = sin.(10x) .+ rand(n)
    return x, y
end
x, y = batch(100);
p1 = scatter(x, y, marker_z=(+), color=:blues, legend=false,
    label="data",
    markersize=3, alpha=0.3);
plot(p1)

using Flux

model = Chain(
    Dense(1, 10, relu),
    Dense(10, 10, relu),
    Dense(10, 10, relu),
    Dense(10, 1,))

#loss(x, y) = Flux.Losses.mse(model(x), y)
loss = (m -> (x, y) -> Flux.Losses.mse(m(x), y))(model)
Flux.train!(loss, Flux.params(model), Vector{Tuple}([(batch(100)) for _ in range(1, length=2000)]), Flux.ADAM(0.01))

ERROR: DimensionMismatch("matrix A has dimensions (10,1), vector B has length 100")
Stacktrace:
  [1] generic_matvecmul!(C::Vector{Float64}, tA::Char, A::Matrix{Float32}, B::Vector{Float64}, _add::LinearAlgebra.MulAddMul{true, true, Bool, Bool})
    @ LinearAlgebra C:\Users\user\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:713
  [2] mul!
    @ C:\Users\user\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:81 [inlined]
  [3] mul!
    @ C:\Users\user\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:275 [inlined]
  [4] *
    @ C:\Users\user\AppData\Local\Programs\Julia-1.7.3\share\julia\stdlib\v1.7\LinearAlgebra\src\matmul.jl:47 [inlined]
  [5] rrule
    @ C:\Users\user\.julia\packages\ChainRules\uh22h\src\rulesets\Base\arraymath.jl:40 [inlined]
  [6] rrule
    @ C:\Users\user\.julia\packages\ChainRulesCore\GUvJT\src\rules.jl:134 [inlined]
  [7] chain_rrule
    @ C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\chainrules.jl:217 [inlined]
  [8] macro expansion
    @ C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0 [inlined]
  [9] _pullback
    @ C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:9 [inlined]
 [10] _pullback
    @ C:\Users\user\.julia\packages\Flux\js6mP\src\layers\basic.jl:159 [inlined]
 [11] _pullback(ctx::Zygote.Context, f::Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, args::Vector{Float64})
    @ Zygote C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [12] macro expansion
    @ C:\Users\user\.julia\packages\Flux\js6mP\src\layers\basic.jl:53 [inlined]
 [13] _pullback
    @ C:\Users\user\.julia\packages\Flux\js6mP\src\layers\basic.jl:53 [inlined]
 [14] _pullback(::Zygote.Context, ::typeof(Flux.applychain), ::Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}, ::Vector{Float64})
    @ Zygote C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [15] _pullback
    @ C:\Users\user\.julia\packages\Flux\js6mP\src\layers\basic.jl:51 [inlined]
 [16] _pullback
    @ f:\projects\Julia Flux\miejemplo.jl:33 [inlined]
 [17] _pullback(::Zygote.Context, ::var"#6#8"{Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}}, ::Vector{Float64}, ::Vector{Float64})
    @ Zygote C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [18] _apply
    @ .\boot.jl:814 [inlined]
 [19] adjoint
    @ C:\Users\user\.julia\packages\Zygote\DkIUK\src\lib\lib.jl:204 [inlined]
 [20] _pullback
    @ C:\Users\user\.julia\packages\ZygoteRules\AIbCs\src\adjoint.jl:65 [inlined]
 [21] _pullback
    @ C:\Users\user\.julia\packages\Flux\js6mP\src\optimise\train.jl:120 [inlined]
 [22] _pullback(::Zygote.Context, ::Flux.Optimise.var"#37#40"{var"#6#8"{Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}}, Tuple{Vector{Float64}, Vector{Float64}}})
    @ Zygote C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface2.jl:0
 [23] pullback(f::Function, ps::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}})
    @ Zygote C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface.jl:352
 [24] gradient(f::Function, args::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}})
    @ Zygote C:\Users\user\.julia\packages\Zygote\DkIUK\src\compiler\interface.jl:75
 [25] macro expansion
    @ C:\Users\user\.julia\packages\Flux\js6mP\src\optimise\train.jl:119 [inlined]
 [26] macro expansion
    @ C:\Users\user\.julia\packages\ProgressLogging\6KXlp\src\ProgressLogging.jl:328 [inlined]
 [27] train!(loss::Function, ps::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}}, data::Vector{Tuple}, opt::ADAM; cb::Flux.Optimise.var"#38#41")
    @ Flux.Optimise C:\Users\user\.julia\packages\Flux\js6mP\src\optimise\train.jl:117
 [28] train!(loss::Function, ps::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}}, data::Vector{Tuple}, opt::ADAM)
    @ Flux.Optimise C:\Users\user\.julia\packages\Flux\js6mP\src\optimise\train.jl:114
 [29] top-level scope
    @ f:\projects\Julia Flux\miejemplo.jl:34
2

Please I need your guidance. I have read and read many sites where a similar error is reported: #ERROR:DimensionMismatch(“matrix A has dimensions (xx,yy), vector B has length zz”)
I just can’t get the logical thinking to solve it.
On the other hand the code runs perfectly from Python (its original version; I just try to translate it to Julia)

"Summary: Example 2: Calling Julia from Python 
import numpy as np, matplotlib.pyplot as plt
from juliacall import Main as jl, convert as jlconvert

def batch(n):
    x = np.sort(np.random.uniform(-1, 1,(1,n)))
    y = np.sin(x*10) + np.random.normal(0, 0.1,(1,n))
    return x, y

jl.seval("using Flux")
model = jl.Chain(
   jl.Dense(1, 10, jl.relu),
   jl.Dense(10, 10, jl.relu),
   jl.Dense(10, 10, jl.relu),
   jl.Dense(10, 1),
)

loss = jl.seval("m -> (x, y) -> Flux.Losses.mse(m(x), y)")(model)

jl.Flux.train_b(
    loss,
    jl.Flux.params(model),
    jlconvert(jl.Vector[jl.Tuple], [batch(100) for _ in range(2000)]),
    jl.Flux.ADAM(0.01),
)

x, y = batch(400)
plt.scatter(x[0], y[0], label="truth")
yhat = model(x)
plt.plot(x[0], yhat[0, :], c="k", label="model")
plt.legend()
print ("loss =", loss(x, y) )
plt.show()

juliacall_from_Python
juliacall_from_Python1265×899 199 KB

I have tried in several ways to achieve the function in Julia:
#jlconvert(jl.Vector[jl.Tuple], [batch(100) for _ in range(2000)])
that my way of inquiring has the type:

 jlconvert(jl.Vector[jl.Tuple], [batch(2) for _ in range(5)])
 jl Tuple[([0.8143505082673181 0.8738088849175676], [0.8580431091654733 0.5895791978138638]), 
          ([-0.9483384509863588 0.47626054348305136], [-0.04851664307944544 -0.8797085687488064]),
          ([-0.5957084756344351 -0.506964544048967], [0.2948794476985104 0.9787866354126213]),
          ([-0.8927181004355458 -0.7424697990014366], [-0.4528059088708106 -1.0321507313265177]),
          ([-0.6219316227190661 -0.32410230150294517], [0.11996326531533408 0.08767804218148574])]>
=#
juliacall = [([0.8143505082673181 0.8738088849175676], [0.8580431091654733 0.5895791978138638]),
    ([-0.9483384509863588 0.47626054348305136], [-0.04851664307944544 -0.8797085687488064]),
    ([-0.5957084756344351 -0.506964544048967], [0.2948794476985104 0.9787866354126213]),
    ([-0.8927181004355458 -0.7424697990014366], [-0.4528059088708106 -1.0321507313265177]),
    ([-0.6219316227190661 -0.32410230150294517], [0.11996326531533408 0.08767804218148574])]
typeof(juliacall)
#Vector{Tuple{Matrix{Float64}, Matrix{Float64}}} (alias for Array{Tuple{Array{Float64, 2}, Array{Float64, 2}}, 1})
typeof(juliacall[1, 1][1])
#Matrix{Float64} (alias for Array{Float64, 2})

And these have been my failed attempts:

Vector{Tuple{Matrix{Float64}, Matrix{Float64}}} 
anytuple_points = Tuple{Any,Any}[batch(2) for _ in 1:5]
typeof(anytuple_points)
#Vector{Tuple{Any, Any}} (alias for Array{Tuple{Any, Any}, 1})

A = Vector{Tuple}([(batch(2)) for _ in range(1, length=5)])
typeof(A)
#Vector{Tuple} (alias for Array{Tuple, 1})


B = [(batch(2)) for _ in range(1, length=5)]
typeof(B)
#Vector{Tuple{Vector{Float64}, Vector{Float64}}} (alias for Array{Tuple{Array{Float64, 1}, Array{Float64, 1}}, 1})

C = [[(batch(2))] for _ in range(1, length=5)]
typeof(C)
#Vector{Vector{Tuple{Vector{Float64}, Vector{Float64}}}} (alias for Array{Array{Tuple{Array{Float64, 1}, Array{Float64, 1}}, 1}, 1})

Following some of the ideas read on the sites(Forums)
I have reproduced some of them to understand how to match the data with the model or the model with the data!!

*How to match the data with the model or the model with the data* 
s = Flux.batch(batch(5))
ss = reshape(s, (length(s), 1))
model(ss)
#ERROR: DimensionMismatch("A has dimensions (10,1) but B has dimensions (10,1)")
sa = reduce(hcat, batch(5))
ssa = reshape(sa, (length(sa), 1))
model(ssa)
#ERROR:DimensionMismatch("A has dimensions (10,1) but B has dimensions (10,1)")

model[1].weight
#10×1 Matrix{Float32}:
model[1].bias
#10-element Vector{Float32}:
Dense(1, 10, relu)(rand(1, 10))
#10×10 Matrix{Float64}:
model[2].weight
#10×10 Matrix{Float32}:
model[2].bias
#10-element Vector{Float32}:
Dense(10, 10, relu)(rand(10, 10))
#10×10 Matrix{Float64}:
model[3].weight
#10×10 Matrix{Float32}:
model[3].bias
#10-element Vector{Float32}:
Dense(10, 10, relu)(rand(10, 10))
#10×10 Matrix{Float64}:
model[4].weight
#1×10 Matrix{Float32}:
model[4].bias
#1-element Vector{Float32}:
model[4].bias
Dense(10, 1, relu)(rand(10, 10))
#1×10 Matrix{Float64}:
Dense(10, 1, relu)(rand(10, 1))
#1×1 Matrix{Float64}:

I would appreciate any help:
The code correction.
Examples where the matching of the model and the data is requested or shown
That sections of the Flux.jl documentation could guide me to the knowledge of the pairing of the data and the model.

:+1:

3

Hi @HerAdri
Your question is very long and i boiled it down to solving this error !!

Your model is defined to take only 1 input feature → 

model = Chain(
    Dense(1, 10, relu),  # 1 input feature --> 10 output features
    Dense(10, 10, relu),
    Dense(10, 10, relu),
    Dense(10, 1,))

but, you are feeding it with 100-element Vector (x) which corresponds to 100 features → 

function batch(n)
    x = sort(rand(-1.1:0.01:1.1, n))
    y = sin.(10x) .+ rand(n)
    return x, y
end
x, y = batch(100);  # 100-element Vectors

So, you need to convert inputs to correct form that Flux expects, i.e, inputs should be of the form (number_of_features, batch_size).
In your case input should be of size (1, 100) which can be done by using Flux.unsqueeze as follows → 

x = Flux.unsqueeze(x, dims=1)  # 1×100 Matrix
2 Likes