# Build a Hybrid Neural Network using Flux

**URL:** <https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349>\
**Category:** General Usage\
**Tags:** question, flux\
**Created:** [February 9, 2023, 2:48pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349 "2023-02-09T14:48:28Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 9, 2023, 2:48pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/1 "2023-02-09T14:48:28Z")

</div>

When using Flux.jl to build neural network, I want to build a hybrid NN.  
I found [`Flux.PairwiseFusion`] ([Built-in Layers · Flux](https://fluxml.ai/Flux.jl/stable/models/layers/#Flux.PairwiseFusion)) can meet my need.  
But when I train the NN:

```julia
using Flux
using Statistics
using Flux: train!

m2 = PairwiseFusion(vcat, Dense(1,1), Dense(2,1))
x1, x2 = hcat(0:4...), hcat(6:10...)
actual(x) = 4x + 2
y = actual.(x1)

data = [((x1,x2), y)]
loss(m, (xa,xb), yt) = mean(abs2.(m(xa,xb)[2].-yt))
loss(m2, (x1,x2), y)

opt = Descent()
data = [(((x1,x2)),y)]
train!(loss, m2, data, opt)

```

Errors are

```julia

MethodError: no method matching train!(::typeof(loss), ::PairwiseFusion{typeof(vcat), Tuple{Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}, ::Vector{Tuple{Tuple{Matrix{Int64}, Matrix{Int64}}, Matrix{Int64}}}, ::Descent)
Closest candidates are:
  train!(::Any, !Matched::Zygote.Params, ::Any, ::Flux.Optimise.AbstractOptimiser; cb) at C:\Users\Lenovo\.julia\packages\Flux\EXOFx\src\optimise\train.jl:113
in eval at base\boot.jl:368 
in top-level scope at test.jl:134

```

What can I do?  
Thanks very much!

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [February 9, 2023, 4:32pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/2 "2023-02-09T16:32:40Z")

</div>

I think you’re using an old version of Flux, but following new docs. Try `] add Flux@0.13.12`

---

<div class="post-metadata">

**Author:** ![trilobit](https://avatars.discourse-cdn.com/v4/letter/t/a8b319/32.png) [@trilobit](https://discourse.julialang.org/u/trilobit)\
**Post date:** [February 9, 2023, 4:46pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/3 "2023-02-09T16:46:46Z")

</div>

This post was temporarily hidden by the community for possibly being off-topic, unfocused, inappropriate, or spammy.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [February 9, 2023, 5:13pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/4 "2023-02-09T17:13:59Z")

</div>

Please don’t post this kind of model-generated output unless you’ve vetted the code yourself as working. In this case the generated example is fundamentally wrong and doesn’t address the original post at all. In fact, it’s worse because there’s the risk this incorrect code will confuse future readers and be spread around by search engines.

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 10, 2023, 12:58am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/5 "2023-02-10T00:58:04Z")

</div>

Sorry to above questions.  
The codes I tried are as follows.

```julia
using Flux
using Statistics
using Flux: train!

m2 = PairwiseFusion(vcat, Dense(1,1), Dense(2,1))
x1, x2 = hcat(0:4...), hcat(6:10...)
actual(x) = 4x + 2
y = actual.(x1)

data = [((x1,x2), y)]
loss(m, (xa,xb), yt) = mean(abs2.(m(xa,xb)[2].-yt))
loss(m2, (x1,x2), y)

opt = Descent()
data = [(((x1,x2)),y)]
train!(loss, m2, data, opt)

```

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 10, 2023, 4:04am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/6 "2023-02-10T04:04:00Z")

</div>

Yes, When I try the same model in docs, there are also errors.  
I am using Flux@0.13.5  
when I update or `] add Flux@0.13.12`  
it gives me an error:

```julia
ERROR: Unsatisfiable requirements detected for package Flux [587475ba]:
 Flux [587475ba] log:
 ├─possible versions are: 0.4.1-0.13.5 or uninstalled
 └─restricted to versions 0.13.12 by an explicit requirement — no versions left

```

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [February 10, 2023, 4:30am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/7 "2023-02-10T04:30:46Z")

</div>

> [@Xuekui\_wang](#):
>
> Sorry to above questions.

Oh I wasn’t referring to your posts, no worries! Can you post the contents of the Project.toml for the environment this is running in? It seems like you’re in a project which has manually pinned a particular Flux version.

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 10, 2023, 7:49am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/8 "2023-02-10T07:49:26Z")

</div>

> [@mcabbott](#):
>
> add Flux@0.13.12

```julia
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
Cairo = "159f3aea-2a34-519c-b102-8c37f9878175"
Compose = "a81c6b42-2e10-5240-aca2-a61377ecd94b"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
FileIO = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
Fontconfig = "186bb1d3-e1f7-5a2c-a377-96d770f13627"
GeometricFlux = "7e08b658-56d3-11e9-2997-919d5b31e4ea"
GraphPlot = "a2cc645c-3eea-5389-862e-a155d0052231"
GraphSignals = "3ebe565e-a4b5-49c6-aed2-300248c3a9c1"
Graphs = "86223c79-3864-5bf0-83f7-82e725a168b6"
Gurobi = "2e9cd046-0924-5485-92f1-d5272153d98b"
Ipopt = "b6b21f68-93f8-5de0-b562-5493be1d77c9"
JLD2 = "033835bb-8acc-5ee8-8aae-3f567f8a3819"
JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
MLDataUtils = "cc2ba9b6-d476-5e6d-8eaf-a92d5412d41d"
MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
MetaGraphs = "626554b9-1ddb-594c-aa3c-2596fe9399a5"
Optim = "429524aa-4258-5aef-a3af-852621145aeb"
OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
Pipe = "b98c9c47-44ae-5843-9183-064241ee97a0"
PowerModels = "c36e90e8-916a-50a6-bd94-075b64ef4655"
PowerModelsACDC = "ff45984e-d068-5f4c-9e32-c4133509d236"
PrettyPrint = "8162dcfd-2161-5ef2-ae6c-7681170c5f98"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"
PyCall = "438e738f-606a-5dbb-bf0a-cddfbfd45ab0"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
Setfield = "efcf1570-3423-57d1-acb7-fd33fddbac46"
Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c"
XLSX = "fdbf4ff8-1666-58a4-91e7-1b58723a45e0"
YAML = "ddb6d928-2868-570f-bddf-ab3f9cf99eb6"

```

by `] status`

```julia
[052768ef] CUDA v3.12.0
  [159f3aea] Cairo v1.0.5
  [a81c6b42] Compose v0.9.4
  [a93c6f00] DataFrames v1.3.4
  [864edb3b] DataStructures v0.18.13
  [31c24e10] Distributions v0.25.70
  [5789e2e9] FileIO v1.15.0
  [587475ba] Flux v0.13.5
  [186bb1d3] Fontconfig v0.4.0
  [7e08b658] GeometricFlux v0.13.6
  [a2cc645c] GraphPlot v0.5.2
  [3ebe565e] GraphSignals v0.7.2
  [86223c79] Graphs v1.7.2
  [2e9cd046] Gurobi v0.11.5 `https://github.com/jump-dev/Gurobi.jl#master`
  [b6b21f68] Ipopt v1.0.4
  [033835bb] JLD2 v0.4.22
  [4076af6c] JuMP v1.2.1
  [cc2ba9b6] MLDataUtils v0.5.4
  [b8f27783] MathOptInterface v1.8.0
  [626554b9] MetaGraphs v0.7.1
  [429524aa] Optim v1.7.2
  [bac558e1] OrderedCollections v1.4.1
  [b98c9c47] Pipe v1.3.0
  [c36e90e8] PowerModels v0.19.6
  [ff45984e] PowerModelsACDC v0.6.1 `https://github.com/Electa-Git/PowerModelsACDC.jl#master`
  [8162dcfd] PrettyPrint v0.2.0
  [92933f4c] ProgressMeter v1.7.2
  [438e738f] PyCall v1.94.1
  [295af30f] Revise v3.4.0
⌅ [efcf1570] Setfield v0.8.2
  [9f7883ad] Tracker v0.2.21
  [fdbf4ff8] XLSX v0.8.3
  [ddb6d928] YAML v0.4.7
  [8ba89e20] Distributed
  [9a3f8284] Random

```

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [February 12, 2023, 5:28pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/9 "2023-02-12T17:28:33Z")

</div>

I see you’ve only posted the dependencies section of your Project.toml. Can you share the whole file? The part I’m most interested in isn’t included.

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 14, 2023, 8:13am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/10 "2023-02-14T08:13:21Z")

</div>

Thank you for your attention, My Flux version is 0.3.5, and the docs is not fitted well.  
`Flux.train!() ` function has been changed.

`train!(loss, pars::Params, data, opt::AbstractOptimiser; [cb])`  
had been changed to  
`train!(loss, model, data, opt_state)`  
and there are many differences, for example: `withgradient`、`Flux.update!` and so on.

Now I am using version 0.3.5 instead of the newest version 0.3.12, I do not know the reason that my computer cannot be installed v0.3.12, maybe gpu is not supported in my pc.

I have solved my problem using `Flux.PairwiseFusion `

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [February 15, 2023, 4:16pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/11 "2023-02-15T16:16:52Z")

</div>

> [@Xuekui\_wang](#):
>
> Now I am using version 0.3.5 instead of the newest version 0.3.12, I do not know the reason that my computer cannot be installed v0.3.12

It’s because of something in your Project.toml, which is why I asked to post the entire contents of that file. Is there something in there you’re not able to share?

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 17, 2023, 1:07am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/12 "2023-02-17T01:07:42Z")

</div>

Thank you for your attention again.  
I am using atom, and not very familiar to julia. In my Project.toml, thres are just these contents.  
When I activate a new environment, also cannot install v0.13.12

 ![image](https://global.discourse-cdn.com/julialang/original/3X/a/9/a949da41fa7d3d037d23d93c5863b7c4f6c52f43.png)  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/a/ba16f09db659bf61c46a744059b3bd9d255653ed.png)

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [February 17, 2023, 4:00pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/13 "2023-02-17T16:00:58Z")

</div>

That’s odd. Can you try creating a temporary environment with `] activate --temp` and then `] add Flux@0.13.12`? If that works then we have a lead, but if it throws an error please copy it here.

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [February 17, 2023, 11:46pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/14 "2023-02-17T23:46:50Z")

</div>

Thank you very much for your help.  
I try it as follws in Win10

 ![image](https://global.discourse-cdn.com/julialang/original/3X/3/a/3a6607741dd99a33d124644af0cb365a2013d01d.png)

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [February 21, 2023, 12:29am UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/15 "2023-02-21T00:29:14Z")

</div>

Ah, looks like you can’t access the registry for some reason. Does this machine have internet access, and if so can you access any package server URL (e.g. [pkg.julialang.org](http://pkg.julialang.org))? [https://status.julialang.org/](https://status.julialang.org/) has a full list, and you can try a couple which are closest to you. Depending on your location, you may also want to see [Julia PkgServer 镜像服务及镜像站索引 - 库的安装及使用 - Julia中文社区](https://discourse.juliacn.com/t/topic/2969).

---

<div class="post-metadata">

**Author:** ![Xuekui\_wang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xuekui_wang/32/30746_2.png) [@Xuekui\_wang](https://discourse.julialang.org/u/Xuekui_wang)\
**Post date:** [March 5, 2023, 12:41pm UTC](https://discourse.julialang.org/t/build-a-hybrid-neural-network-using-flux/94349/16 "2023-03-05T12:41:01Z")

</div>

Thank you very much, my friend. It is a great help for me to use Julia Pkg.  
Yes, use PkgServer, everything is Ok, and now Flux.jl@0.13.13 has been installed.