Need to include one file multiple times, how to avoid warning

I get some warnings when I include one public file in several other files in a same package. It is straightforward to include the public file which defines some common functions. The code is as follows.

# ./src/test_redefine.jl 
module test_redefine

export Dog, my_add 

include("utils.jl")         #include utils.jl the first time
include("dog.jl")         
end # module

# ./src/dog.jl
include("utils.jl")    #include utils.jl the second time

struct Dog
    name::String
end

# ./src/utils.jl
function my_add(a, b) 
    println(a, b)
end

in the test file

#./test/runtests.jl
using test_redefine
my_add(1,1)

Warning is as follows,

WARNING: Method definition my_add(Any, Any) in module test_redefine at /Users/zhangliye/code/test_redefine/src/utils.jl:2 overwritten at /Users/zhangliye/code/test_redefine/src/utils.jl:2.
11
   Testing test_redefine tests passed 

In c/c++, it is very straightforward to define some command functions in a public file and include it in the files using it.

Julia version,

Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-8500 CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
1 Like

The problem stems from exporting my_add from inside the module test_redefine back to Main.
When you include utils.jl in Main for the second time, it conflicts with the exported my_add.
Since both versions of my_add are the same, deleting the export should do the trick.

Thank you so much for your response. I know the reason for this. I think this is very straightforward to include the public file in all files using it. Why Julia still give the warning? In C++/C we include the head file in files using it, and when compile the code only one function is defined. I want to know if Julia can do the same or just do not give the warning if the source file and function are the same.

1 Like

Put your “utils” into a module and use it with “using utils”.

You should not need to include the same file multiple times in Julia, so the answer to your question is “don’t do that”. This is an important difference between the way Julia and C++ are used. Instead, you can do:

module A

include("B.jl")
using .B

module C
  using ..B
end

end

Where B.jl looks like:

module B
...
end

The using ..B line lets you load the B module from A inside C without having to include it twice.

4 Likes

Alternatively you can add your current path to your module search path:

push!(LOAD_PATH, pwd())

Then you don’t need any include, just use “using”.

Thank you so much! It can be solved by put the file into one module. However, the code look not so clean. The two version comparison is as follows,
Version 1: Put the file into one module

# file1: ./src/test_redefine.jl 
module test_redefine

export Dog, my_add 

include("utils.jl")
using .utils                  # one more code here
include("dog.jl")
end # module

# file2: ./src/dog.jl
using .utils      # changed to us module, the user should pay attention the definition order, `utils` must defined ahead

struct Dog
    name::String
end

# file3: ./src/utils.jl
module utils         # one more line code here
export my_add    # one more code here

    function my_add(a, b) 
        println("my_add ", a, b)
    end
end       # one more code here 

Version 2: Just do as in C++/C, include the public head file. The code is more cleaner and more straightforward.

# file1: ./src/test_redefine.jl 
module test_redefine

export Dog, my_add 

include("utils.jl")         #include utils.jl the first time
include("dog.jl")         
end # module

# file2: ./src/dog.jl
include("utils.jl")    #include utils.jl the second time

struct Dog
    name::String
end

# file3: ./src/utils.jl
function my_add(a, b) 
    println(a, b)
end
1 Like

In utils.jl I define some enum variables which is used in several files as follows

# utils.jl
@enum State waiting=1 started=2 arrived=3

It is not logically reasonable to put it in a separate module.

Your second way is cleaner. However, I do not think you need to include("utils.jl") twice. I prefer modify your first way to

# file1: ./src/test_redefine.jl
module test_redefine

using Reexport: @reexport  # @reexport does what you do `export my_add` there

include("utils.jl")
@reexport using .utils

include("dog.jl")

end # module

Is it really unclean? I think it is pretty good.

# file2: ./src/dog.jl
using .utils      # changed to us module, the user should pay attention the definition order, `utils` must defined ahead

export Dog  # I prefer to export each type/method at the places where it is defined

struct Dog
    name::String
end

The downside is, you cannot run this file directly. Every time you have to using .test_redefine, then Dog is loaded. I think this is the “correct” way when you are writing a package.

# file3: ./src/utils.jl
module utils         # one more line code here
export my_add    # one more code here

function my_add(a, b)
    println("my_add ", a, b)
end

end       # one more code here

Here I used Reexport.jl.

2 Likes

I don’t see anything wrong with including a simple file that just contains some constants multiple times. Am I missing any issues this could generate?

As for the reexport solution: I find it a bit complicated for the use case here. The way I understand it, @zhangliye just wants to export a single object from utils.jl. So why not simply:

module test_redefine

export my_add
include("utils.jl")
include("dog.jl")

end # module

Thanks for your reply. There is no error, just a lot of warnings. When the project is large, there are too many warnings as follows.

WARNING: Method definition my_add(Any, Any) in module test_redefine at /Users/zhangliye/code/test_redefine/src/utils.jl:2 overwritten at /Users/zhangliye/code/test_redefine/src/utils.jl:2.
11
   Testing test_redefine tests passed 

If you really want to stick with your original code structure, you can get rid of the warnings by just not exporting my_add twice, i.e., by not exporting it from test_redefine.

It’s definitely not recommended in Julia. Any types or functions defined in that file will be defined twice, and those two definitions will be completely different, despite having the same name. This can lead to confusing situations. For some examples of real-world problems caused by multiple includes, see:

3 Likes

When the project is large, there are too many warnings as follows.

Yes, these warnings are important, and they are trying to tell you that by calling include() on the same file multiple times you are re-defining the same function over and over, which is at best wasteful and at worst can lead to problems like Trouble importing a module that depends on another globally imported module

2 Likes

Thank you so much! Extremely useful information!

1 Like