Build a Hybrid Neural Network using Flux

General Usage
,
1

When using Flux.jl to build neural network, I want to build a hybrid NN.
I found [Flux.PairwiseFusion] (Built-in Layers · Flux) can meet my need.
But when I train the NN:

using Flux
using Statistics
using Flux: train!

m2 = PairwiseFusion(vcat, Dense(1,1), Dense(2,1))
x1, x2 = hcat(0:4...), hcat(6:10...)
actual(x) = 4x + 2
y = actual.(x1)

data = [((x1,x2), y)]
loss(m, (xa,xb), yt) = mean(abs2.(m(xa,xb)[2].-yt))
loss(m2, (x1,x2), y)

opt = Descent()
data = [(((x1,x2)),y)]
train!(loss, m2, data, opt)

Errors are


MethodError: no method matching train!(::typeof(loss), ::PairwiseFusion{typeof(vcat), Tuple{Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}, ::Vector{Tuple{Tuple{Matrix{Int64}, Matrix{Int64}}, Matrix{Int64}}}, ::Descent)
Closest candidates are:
  train!(::Any, !Matched::Zygote.Params, ::Any, ::Flux.Optimise.AbstractOptimiser; cb) at C:\Users\Lenovo\.julia\packages\Flux\EXOFx\src\optimise\train.jl:113
in eval at base\boot.jl:368 
in top-level scope at test.jl:134

What can I do?
Thanks very much!

2

I think you’re using an old version of Flux, but following new docs. Try ] add Flux@0.13.12

1 Like
4

Please don’t post this kind of model-generated output unless you’ve vetted the code yourself as working. In this case the generated example is fundamentally wrong and doesn’t address the original post at all. In fact, it’s worse because there’s the risk this incorrect code will confuse future readers and be spread around by search engines.

5 Likes
5

Sorry to above questions.
The codes I tried are as follows.

using Flux
using Statistics
using Flux: train!

m2 = PairwiseFusion(vcat, Dense(1,1), Dense(2,1))
x1, x2 = hcat(0:4...), hcat(6:10...)
actual(x) = 4x + 2
y = actual.(x1)

data = [((x1,x2), y)]
loss(m, (xa,xb), yt) = mean(abs2.(m(xa,xb)[2].-yt))
loss(m2, (x1,x2), y)

opt = Descent()
data = [(((x1,x2)),y)]
train!(loss, m2, data, opt)
6

Yes, When I try the same model in docs, there are also errors.
I am using Flux@0.13.5
when I update or ] add Flux@0.13.12
it gives me an error:

ERROR: Unsatisfiable requirements detected for package Flux [587475ba]:
 Flux [587475ba] log:
 ├─possible versions are: 0.4.1-0.13.5 or uninstalled
 └─restricted to versions 0.13.12 by an explicit requirement — no versions left
7

Oh I wasn’t referring to your posts, no worries! Can you post the contents of the Project.toml for the environment this is running in? It seems like you’re in a project which has manually pinned a particular Flux version.

8 
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
Cairo = "159f3aea-2a34-519c-b102-8c37f9878175"
Compose = "a81c6b42-2e10-5240-aca2-a61377ecd94b"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
FileIO = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
Fontconfig = "186bb1d3-e1f7-5a2c-a377-96d770f13627"
GeometricFlux = "7e08b658-56d3-11e9-2997-919d5b31e4ea"
GraphPlot = "a2cc645c-3eea-5389-862e-a155d0052231"
GraphSignals = "3ebe565e-a4b5-49c6-aed2-300248c3a9c1"
Graphs = "86223c79-3864-5bf0-83f7-82e725a168b6"
Gurobi = "2e9cd046-0924-5485-92f1-d5272153d98b"
Ipopt = "b6b21f68-93f8-5de0-b562-5493be1d77c9"
JLD2 = "033835bb-8acc-5ee8-8aae-3f567f8a3819"
JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
MLDataUtils = "cc2ba9b6-d476-5e6d-8eaf-a92d5412d41d"
MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
MetaGraphs = "626554b9-1ddb-594c-aa3c-2596fe9399a5"
Optim = "429524aa-4258-5aef-a3af-852621145aeb"
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
Pipe = "b98c9c47-44ae-5843-9183-064241ee97a0"
PowerModels = "c36e90e8-916a-50a6-bd94-075b64ef4655"
PowerModelsACDC = "ff45984e-d068-5f4c-9e32-c4133509d236"
PrettyPrint = "8162dcfd-2161-5ef2-ae6c-7681170c5f98"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
Setfield = "efcf1570-3423-57d1-acb7-fd33fddbac46"
Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c"
XLSX = "fdbf4ff8-1666-58a4-91e7-1b58723a45e0"
YAML = "ddb6d928-2868-570f-bddf-ab3f9cf99eb6"

by ] status

[052768ef] CUDA v3.12.0
  [159f3aea] Cairo v1.0.5
  [a81c6b42] Compose v0.9.4
  [a93c6f00] DataFrames v1.3.4
  [864edb3b] DataStructures v0.18.13
  [31c24e10] Distributions v0.25.70
  [5789e2e9] FileIO v1.15.0
  [587475ba] Flux v0.13.5
  [186bb1d3] Fontconfig v0.4.0
  [7e08b658] GeometricFlux v0.13.6
  [a2cc645c] GraphPlot v0.5.2
  [3ebe565e] GraphSignals v0.7.2
  [86223c79] Graphs v1.7.2
  [2e9cd046] Gurobi v0.11.5 `https://github.com/jump-dev/Gurobi.jl#master`
  [b6b21f68] Ipopt v1.0.4
  [033835bb] JLD2 v0.4.22
  [4076af6c] JuMP v1.2.1
  [cc2ba9b6] MLDataUtils v0.5.4
  [b8f27783] MathOptInterface v1.8.0
  [626554b9] MetaGraphs v0.7.1
  [429524aa] Optim v1.7.2
  [bac558e1] OrderedCollections v1.4.1
  [b98c9c47] Pipe v1.3.0
  [c36e90e8] PowerModels v0.19.6
  [ff45984e] PowerModelsACDC v0.6.1 `https://github.com/Electa-Git/PowerModelsACDC.jl#master`
  [8162dcfd] PrettyPrint v0.2.0
  [92933f4c] ProgressMeter v1.7.2
  [438e738f] PyCall v1.94.1
  [295af30f] Revise v3.4.0
⌅ [efcf1570] Setfield v0.8.2
  [9f7883ad] Tracker v0.2.21
  [fdbf4ff8] XLSX v0.8.3
  [ddb6d928] YAML v0.4.7
  [8ba89e20] Distributed
  [9a3f8284] Random
9

I see you’ve only posted the dependencies section of your Project.toml. Can you share the whole file? The part I’m most interested in isn’t included.

10

Thank you for your attention, My Flux version is 0.3.5, and the docs is not fitted well.
Flux.train!() function has been changed.

train!(loss, pars::Params, data, opt::AbstractOptimiser; [cb])
had been changed to
train!(loss, model, data, opt_state)
and there are many differences, for example: withgradient、Flux.update! and so on.

Now I am using version 0.3.5 instead of the newest version 0.3.12, I do not know the reason that my computer cannot be installed v0.3.12, maybe gpu is not supported in my pc.

I have solved my problem using Flux.PairwiseFusion

11

It’s because of something in your Project.toml, which is why I asked to post the entire contents of that file. Is there something in there you’re not able to share?

12

Thank you for your attention again.
I am using atom, and not very familiar to julia. In my Project.toml, thres are just these contents.
When I activate a new environment, also cannot install v0.13.12

image
image1080×827 147 KB

image
image1082×805 135 KB

13

That’s odd. Can you try creating a temporary environment with ] activate --temp and then ] add Flux@0.13.12? If that works then we have a lead, but if it throws an error please copy it here.

14

Thank you very much for your help.
I try it as follws in Win10

image
image1319×536 53.2 KB

15

Ah, looks like you can’t access the registry for some reason. Does this machine have internet access, and if so can you access any package server URL (e.g. pkg.julialang.org)? https://status.julialang.org/ has a full list, and you can try a couple which are closest to you. Depending on your location, you may also want to see Julia PkgServer 镜像服务及镜像站索引 - 库的安装及使用 - Julia中文社区.

1 Like
16

Thank you very much, my friend. It is a great help for me to use Julia Pkg.
Yes, use PkgServer, everything is Ok, and now Flux.jl@0.13.13 has been installed.

1 Like