Using XY in module Main conflicts with an existing identifier - proper way of loading modules in REPL?

General Usage
1

I’ve been learning Julia primarily by writing some Evolutionary Algo code, I started ~1 year ago, when I wrote my first, very simple, very primitive EA lib which I decided to re-write after a year (haven’t touched Julia ever since), however I stumbled upon weird behavior which I didn’t really encounter ~1 year ago when I wrote the original version (not sure if this happens in the original version, I’d assume not because of reasons mentioned below)

as the code is messy and I’m just learning, I will provide only simplified version of the Subject and EA modules, and a testing script

basically, I have a subject module: which defines abstract subject, binary subject structure, and functions that can work with binary subject (getters/setters for genes and fitness), I also have a EA module: which defines abstractAlgorithm and concrete algorithm structure (remapping, selection, etc functions) and provides 2 functions that work with abstract algorithm - evolve() and finished(), these functions are used in the main loop

everything works fine, however the way I use this lib + scripts is that I open the Julia REPL and include the test.jl file, which loads all the libs and runs a test… this works fine the first time, however as soon as I run the include command second time, I get this error

julia> include("test.jl")
done!

julia> include("test.jl")
WARNING: replacing module Subject.
WARNING: replacing module EvolutionaryAlgorithm.
WARNING: using EvolutionaryAlgorithm.Algorithm in module Main conflicts with an existing identifier.
WARNING: using EvolutionaryAlgorithm.finished in module Main conflicts with an existing identifier.
WARNING: using Subject.BinarySubject in module Main conflicts with an existing identifier.
ERROR: LoadError: MethodError: no method matching genes(::BinarySubject)
Closest candidates are:
  genes(::Main.Subject.BinarySubject, ::Array{Bool,1}) at XYZ\subject.jl:14
  genes(::Main.Subject.BinarySubject) at XYZ\subject.jl:13

apparently, after loading it for the second time, the exported struct BinarySubject is no longer BinarySubject, but a Main.Subject.BinarySubject

what am I doing wrong? I assume it’s because I am using .Subject in both, EA module and testing script, but this is something I can’t change

subject.jl:

module Subject
	export AbstractSubject, BinarySubject

	abstract type AbstractSubject end

	mutable struct BinarySubject <: AbstractSubject
		genes::Array{Bool, 1}
		fitness::Float64
		
		BinarySubject(genes::Array{Bool, 1}, fitness::Float64) = new(genes, fitness)
	end

	genes(subject::BinarySubject) = subject.genes
	genes(subject::BinarySubject, genes::Array{Bool, 1}) = subject.genes = genes

	fitness(subject::BinarySubject) = subject.fitness
	fitness(subject::BinarySubject, fitness::Float64) = subject.fitness = fitness
end

algorithm.jl:

include("subject.jl")

module EvolutionaryAlgorithm using ..Subject
	export Algorithm, evolve, finished

	abstract type AbstractAlgorithm end

	struct Algorithm <: AbstractAlgorithm
		stop::Function
	end
	stop(algo::Algorithm) = algo.stop

	# function to evolve subjects
	function evolve(algo::AbstractAlgorithm, subjects::Array{<:AbstractSubject, 1})
		# evolve subjects
	end

	# function that determines whether EA should be stopped
	i = 0 # temporary
	function finished(algo::AbstractAlgorithm, subjects::Array{<:AbstractSubject, 1})
		global i
		stop(algo)(subjects) || (i += 1) >= 10
	end
end

test.jl:

include("algorithm.jl")

using .EvolutionaryAlgorithm, .Subject

function main()
	function fitness(subjects::Array{BinarySubject, 1})
		f(subject::BinarySubject) = Float64(sum(Subject.genes(subject)))
		Subject.fitness.(subjects, f.(subjects))

		sort(subjects, by = (x) -> Subject.fitness(x))
	end
	fitness!(subjects::Array{BinarySubject, 1}) = subjects[:] = fitness(subjects)

	subjects::Array{BinarySubject, 1} = [BinarySubject(rand(Bool, 10), 0.0) for i in 1:100] # create 100 random binary subjects with length 10
	fitness!(subjects)

	stopper(subjects::Array{BinarySubject, 1}) = Subject.fitness(subjects[end]) == 10 # evolution is finished when a subject has all bits set to 1
	algo = Algorithm(stopper)

	while !finished(algo, subjects)
		evolve(algo, subjects)
		fitness!(subjects)
	end

	println("done!")
end

main()
2

The using statement binds constants in global scope. If you wrap the contents of test.jl in a module then including it repeatedly replaces the test module with a new one that has its own global context, so there’s no collision.

3

how exactly should I do this?

I tried to wrap the main function in module TestEA and export main

include("algorithm.jl")

module TestEA
using ..EvolutionaryAlgorithm, ..Subject
export main

function main()
	# main body
end

end

using .TestEA
main()

which worked and I now can invoke include(“test.jl”) as many times as I want, but any changes done to the test module are not being propagated - e.g. changing the ‘done’ output to anything else still results in printing ‘done’

julia> include("test.jl")
WARNING: replacing module Subject.
WARNING: replacing module EvolutionaryAlgorithm.
WARNING: replacing module TestEA.
WARNING: using TestEA.main in module Main conflicts with an existing identifier.
done!

I didn’t have this problem a year ago in a version 1.2.1 or whatever it was

4

The include(algorithm.jl) should also be inside the test module. Don’t put using .TestEA in Main (the REPL context) - that’s the problem I mentioned before. Then if you invoke TestEA.main() it will always be the current one.

5

while this works, it seems like a tedious, and tad dirty way to do it (as a lot of other things in Julia, e.g. composition instead of inheritance) - is this really the best/proper way to do it? or is there other, ‘Julia’ way to test modules/write simple applications?

6

Most people use the Revise package for this kind of development now,
Module juggling is the older approach and was (may still be?) more robust to changes of type design.

2 Likes
7

I agree… make your set of modules a package (not necessarily for public disclosure, can be a private, local package. Basically a package is a module or a collection of modules with the entry point being a .jl file with the name of the package and a Project.toml describing the requirements), be sure that the code you include is included only once and use revise before loading your package so that new modifications you make to the package are accounted in the working session.

1 Like
8

Worth noting, though: if you change the definition of a struct, you still have to restart Julia. This can be quite tedious.

One approach that I like is to develop concurrently with tests. Since Pkg.test runs in a separate environment, one can change struct definitions and constants without having to restart anything.

1 Like