I am attempting to use the same function forward in different contexts.
Layers.jl
module Layers
export Dense, forward
struct Dense
n_inputs
n_neurons
weights
biases
function Dense(n_inputs, n_neurons)
### Intialize weights and biases
...
new(n_inputs, n_neurons, weights, biases)
end
end
function forward(layer::Dense, inputs)
if layer.n_inputs != size(inputs, 2)
throw(DimensionMismatch("Input size does not match layer input size"))
end
return (inputs * layer.weights) .+ layer.biases
end
end
Activations.jl
module Activations
export forward, ReLU
abstract type ReLU end
function forward(layer::Type{ReLU}, data)
return map(x -> max(0, x), data)
end
end
main.jl
using .Layers
using .Activations
dense1 = Dense(2, 4)
inputs = [[1 2]; [3 4]; [3 5]]
out = forward(dense1, inputs)
forward(ReLU, out)
On the last line, I get an error saying that forward does not have an overload for ReLU:
julia> forward(ReLU, out)
ERROR: MethodError: no method matching forward(::Type{ReLU}, ::Array{Float64,2})
Closest candidates are:
forward(::Dense, ::Any) at /Users/user/Documents/test/Layers.jl:33
Stacktrace:
[1] top-level scope at REPL[106]:
Is there a correct way to pull in functions with the same name from multiple different files to use multiple dispatch?
The problem is that you have 2 different functions named forward instead of 1 function with 2 methods. The solution is to make one (or both) of Layers and Activations import forward before defining it.
I think that deserves a design explanation. Naively I would expect that to work and to raise a warning if a method overwrites the other.
Of course for third party modules the alternative is to import. But it is somewhat scary that that kind of overwrite can happen without notice when loading multiple modules.
Hi, thanks for your reply. Iām still having a bit of an issue doing it this way. I tried following this and reading this documentation, yet Iām still unsure as to where Iām going wrong.
New file NNFS.jl
module NNFS
export forward
forward(layer, data) = begin
throw("You need to overload base forward for $(typeof(layer)) and $(typeof(data))")
end
end
Modified Layers.jl
module Layers
using Random; Random.seed!(0)
include("NNFS.jl")
import .NNFS.forward
struct Dense
n_inputs
n_neurons
weights
biases
function Dense(n_inputs, n_neurons)
### Initialize weights and biases
...
new(n_inputs, n_neurons, weights, biases)
end
end
NNFS.forward(layer::Dense, inputs::Array{Float64,2}) = begin
if layer.n_inputs != size(inputs, 2)
throw(DimensionMismatch("Input size does not match layer input size"))
end
return (inputs * layer.weights) .+ layer.biases
end
export Dense
end
Main file:
include("NNFS.jl")
include("Layers.jl")
using .NNFS: forward
using .Layers
dense1 = Dense(2, 4)
inputs = [[1. 2.]; [3. 4.]; [3. 5.]]
out = forward(dense1, inputs)
The last line gets the error (which I defined)
ERROR: "You need to overload base forward for Dense and Array{Float64,2}"
Stacktrace:
[1] forward(::Dense, ::Array{Float64,2}) at /Users/user/test/NNFS.jl:6
[2] top-level scope at REPL[7]:1
And calling methods on forward gives the following
julia> methods(forward)
# 1 method for generic function "forward":
[1] forward(layer, data) in Main.NNFS at /Users/user/test/NNFS.jl:5
What is the correct way to set up function overloading from multiple different files?
Thereās no overwriting going onāthere are just two different functions: Layers.forward and Activations.forward which have nothing whatsoever to do with one another. Neither interferes with the other, but they canāt both participate in multiple dispatch because they are fundamentally different functions.
The reason for this is that making a new function by default means you only need to be aware of the functions you extend. If a new method was created by default, any package (or Base) adding a new function would be a potentially breaking change.
The problem in your case is that youāve now included NNFS.jl in two different placesāonce inside Layers.jl and again in your main file. Those separate include()s have created two completely unrelated modules which happen to have the same name. Thatās why itās important not to include the same file multiple times. Hereās a post with a similar question on avoiding multiple includes that you might find helpful: Need to include one file multiple times, how to avoid warning - #5 by rdeits
I am sorry, there is indeed an error raised, so that is fine:
julia> module Pkg1
f(x) = 1
export f
end
Main.Pkg1
julia> module Pkg2
f(x) = 2
export f
end
Main.Pkg2
julia> using .Pkg1; using .Pkg2
julia> f(1)
WARNING: both Pkg2 and Pkg1 export "f"; uses of it in module Main must be qualified
ERROR: UndefVarError: f not defined
Stacktrace:
[1] top-level scope at REPL[5]:1
julia>
My first impression from the original post was that didnāt raise any error or warning.
Also, I do not understand how did you get the no method matching forward... error. You should have gotten a warning on the impossibility of using forward from two modules without explicitly importing them, and then forward not defined errors. Here is what happens here, reproducing your initial post with some simplifications:
julia> module Layers
export Dense, forward
struct Dense x end
forward(x::Dense) = "from Layers"
end
Main.Layers
julia> module Activations
export ReLU, forward
abstract type ReLU end
forward(x::Type{ReLU}) = "from Activations"
end
Main.Activations
julia> using .Layers
julia> using .Activations
julia> dense1 = Dense(1)
Dense(1)
julia> forward(dense1)
WARNING: both Activations and Layers export "forward"; uses of it in module Main must be qualified
ERROR: UndefVarError: forward not defined
Stacktrace:
[1] top-level scope at REPL[6]:1
julia> forward(ReLU)
ERROR: UndefVarError: forward not defined
Stacktrace:
[1] top-level scope at REPL[7]:1
julia>
As you see, you are overloading the forward function from NNFS, but that is visible only on the NNFS version which lives inside Layers. There are two versions of NNFS, because of the double include, as explained.
I recommend against using submodules, for the majority of use cases. (not all, but most)
It leads too problems like this too easily.
Either use seperate packages and load them via using or import.
Or use seperate files and include them, with only the main file having a module declaration.
Not seperate files containing modules includeed into each other.
At least initially, just put everything in one big namespace.
Namespaces are over-rated, lets have less of them.
This may be counter to your experience in other languages (it was for me), but julia is a different language to others.
And submodules just allow for extra mistakes, and require extra book-keeping.
You can always later decide to split things into seperate packages (or seperate submodules if you really want), but until you have a reason to, just put it all in the same namespace.
I think your comment could be summarized to āusing submodules is non-trivial/āvery different from other languagesā so beginners should avoid it if possibleā. Obviously, any language feature may be wrongly used or overused. However, without talking about specifics it is impossible to draw the line. I use multiple submodules (with āseperate files containing modules includeed each otherā) and have yet to experience any problems regarding this. Nor I have any reason to break my package into multiple packages.
One of my favourite quotes is from the introduction of
Press, Teukolsky, Vetting and Flanneryās āNumerical Recipesā:
We do, therefore, offer you our practical judgements wherever we can. As you gain experience, you will form your own opinion of how reliable our advice is. Be assured that it is not perfect!
This is my advise, your mileage may vary.
Be assured that there are many viable ways to do things, and this is just the one I personally recommend.