I have a crazy question/proposal.
Currently, deepcopy isn’t defined for Modules:
julia> deepcopy(MyModule)
ERROR: deepcopy of Modules not supported
I think it might not be too hard to implement this. I wrote a somewhat reasonable starter implementation of it in Julia (which I’ll include at the end of this post), but it would probably be best implemented at least in part in C.
As far as I can tell, there wasn’t much discussion of supporting this when deepcopy was written; it just seemed awkward to do, so it was left out:
https://github.com/JuliaLang/julia/pull/1242#issuecomment-8212530
So the next question is whether this is worth anything to support. The main motivation I see for deepcopy(::Module) being useful is for mocking for unit tests. Invenia already has a great package for this (GitHub - invenia/Mocking.jl: Allows Julia function calls to be temporarily overloaded for purpose of testing), but it is an intrusive interface (requiring modification of source code), so it doesn’t seem to be widely used.
However, if you can make a copy of a module, you can happily replace the definitions of methods inside that module, and then run your tests in the copy of the module. This actually makes for a very nice interface for mocking!
Here is one example of what such a mocking interface could look like:
module MTest
using ModuleMocks
using Test
module M
foo() = 5
bar() = foo()
end
# Normally, bar() returns 5
@test M.bar() == 5
ModuleMocks.@mock_in_module(M, # (module to mock)
begin # (setup phase)
foo() = 3
end,
begin # (test phase)
# But in this mocked-up module, bar() returns 3
@test M.bar() == 3
end)
# Outside, bar() still returns 5
@test M.bar() == 5
end # module MTest
The hardest part, I think, would be actually deep-copying functions and replacing their calls to other functions in the same module to instead point to the copied functions in the new module. But I think this could be done with a bit of effort.
For now, I was able to get something like the above mocking interface working by just spawning a new worker process, loading the module I want, and making the setup modifications there.
That code currently looks like this:
using Distributed
macro mock_test(setup, test)
quote
(p,) = $Distributed.addprocs(1)
fetch(@spawnat p $Base.eval($(QuoteNode(setup))))
fetch(@spawnat p $Base.eval($(QuoteNode(test))))
Distributed.rmprocs(p)
end
end
# ...
@mock_test(
begin # setup
using Test; using Pkg; Pkg.activate(".")
include("file/to/test.jl")
M.foo() = 3 # (override foo with a mock)
nothing
end,
@testset "M" begin # test
M.bar() == 3
end)
But having to spawn processes is cumbersome and maybe expensive, and it can be difficult to get the right code loaded. It feels like a bit of overkill for mock testing.
Finally, here’s a rough-draft start to some code for deepcopying modules. It’s pretty hacky, and doesn’t work for mocking (b/c of the copied functions still pointing to the old module), but I think this could be cleaned up and improved if some of the logic was moved to C:
# ------------ MODULES: Custom implementation of deepcopy(m::Module) -----------------------
function Base.deepcopy(m::Module)
m2 = Module(nameof(m), true)
import_names_into(m2, m)
setparent!(m2, parentmodule(m))
return m2
end
usings(m::Module) =
ccall(:jl_module_usings, Array{Module,1}, (Any,), m)
function import_names_into(destmodule, srcmodule)
fullsrcname = Meta.parse(join(fullname(srcmodule), "."))
for m in usings(srcmodule)
full_using_name = Expr(:., fullname(m)...)
Core.eval(destmodule, Expr(:using, full_using_name))
end
for n in names(srcmodule, all=true, imported=true)
# don't copy itself into itself; don't copy `include`, define it below instead
if n != nameof(destmodule) && n != :include
srcval = Core.eval(srcmodule, n)
deepcopy_value(destmodule, srcmodule, n, srcval)
end
end
@eval destmodule include(fname::AbstractString) = Main.Base.include(@__MODULE__, fname)
end
# Make new names w/ a copy of the value. For functions, make a new function object.
# TODO: Currently this just makes new function objects, with methods that just call the
# corresponding method in the old module. For mocking, they would need to have the source
# code copied over as well, and modified to call the corresponding function in the new module.
deepcopy_value(destmodule, srcmodule, name, value) = Core.eval(destmodule, :($name = $(deepcopy(value)))
function deepcopy_value(destmodule, srcmodule, name, value::Function)
@eval destmodule function $name end
src_ms = Core.eval(srcmodule, :(methods($value).ms))
for m in src_ms
argnames = collect(Symbol("_$i") for i in 1:m.nargs-1)
argsigs = collect(:($(argnames[i])::$(m.sig.parameters[i+1])) for i in 1:m.nargs-1)
@eval destmodule $name($(argsigs...)) = $srcmodule.$name($(argnames...))
end
end
# EWWWWWWWW, this is super hacky. This would need to be an additional C API either to allow
# setting a module's Parent, or to allow constructing a module with a parent.
setparent!(m::Module, p::Module) =
unsafe_store!(Ptr{_Module2}(pointer_from_objref(m)),
_Module2(nameof(m), p), 1)
struct _Module2
name
parent
end
# ------------------------------------------------------------------------------------------
What do people think? Would something like this be useful and worth pursuing?
If not, is the distributed mocking approach cool, and should we investigate that further?
Thanks!
