`deepcopy(::Module)` and mocking for tests

I have a crazy question/proposal.

Currently, deepcopy isn’t defined for Modules:

julia> deepcopy(MyModule)
ERROR: deepcopy of Modules not supported

I think it might not be too hard to implement this. I wrote a somewhat reasonable starter implementation of it in Julia (which I’ll include at the end of this post), but it would probably be best implemented at least in part in C.

As far as I can tell, there wasn’t much discussion of supporting this when deepcopy was written; it just seemed awkward to do, so it was left out:


So the next question is whether this is worth anything to support. The main motivation I see for deepcopy(::Module) being useful is for mocking for unit tests. Invenia already has a great package for this (https://github.com/invenia/Mocking.jl), but it is an intrusive interface (requiring modification of source code), so it doesn’t seem to be widely used.

However, if you can make a copy of a module, you can happily replace the definitions of methods inside that module, and then run your tests in the copy of the module. This actually makes for a very nice interface for mocking!

Here is one example of what such a mocking interface could look like:

module MTest
using ModuleMocks
using Test

module M
foo() = 5
bar() = foo()
end

# Normally, bar() returns 5
@test M.bar() == 5

ModuleMocks.@mock_in_module(M,  # (module to mock)
    begin  # (setup phase)
        foo() = 3
    end,
    begin  # (test phase)
        # But in this mocked-up module, bar() returns 3
        @test M.bar() == 3
    end)

# Outside, bar() still returns 5
@test M.bar() == 5

end  # module MTest

The hardest part, I think, would be actually deep-copying functions and replacing their calls to other functions in the same module to instead point to the copied functions in the new module. But I think this could be done with a bit of effort.


For now, I was able to get something like the above mocking interface working by just spawning a new worker process, loading the module I want, and making the setup modifications there.

That code currently looks like this:

using Distributed
macro mock_test(setup, test)
    quote
        (p,) = $Distributed.addprocs(1)

        fetch(@spawnat p $Base.eval($(QuoteNode(setup))))
        fetch(@spawnat p $Base.eval($(QuoteNode(test))))

        Distributed.rmprocs(p)
    end
end

# ...

@mock_test(
    begin  # setup
        using Test; using Pkg; Pkg.activate(".")
        include("file/to/test.jl")      
        M.foo() = 3  # (override foo with a mock)
        nothing
    end,
    @testset "M" begin  # test
       M.bar() == 3
    end)

But having to spawn processes is cumbersome and maybe expensive, and it can be difficult to get the right code loaded. It feels like a bit of overkill for mock testing.


Finally, here’s a rough-draft start to some code for deepcopying modules. It’s pretty hacky, and doesn’t work for mocking (b/c of the copied functions still pointing to the old module), but I think this could be cleaned up and improved if some of the logic was moved to C:

# ------------ MODULES: Custom implementation of deepcopy(m::Module) -----------------------
function Base.deepcopy(m::Module)
    m2 = Module(nameof(m), true)
    import_names_into(m2, m)
    setparent!(m2, parentmodule(m))
    return m2
end


usings(m::Module) =
    ccall(:jl_module_usings, Array{Module,1}, (Any,), m)

function import_names_into(destmodule, srcmodule)
    fullsrcname = Meta.parse(join(fullname(srcmodule), "."))
    for m in usings(srcmodule)
        full_using_name = Expr(:., fullname(m)...)
        Core.eval(destmodule, Expr(:using, full_using_name))
    end
    for n in names(srcmodule, all=true, imported=true)
        # don't copy itself into itself; don't copy `include`, define it below instead
        if n != nameof(destmodule) && n != :include
            srcval = Core.eval(srcmodule, n)
            deepcopy_value(destmodule, srcmodule, n, srcval)
        end
    end
    @eval destmodule include(fname::AbstractString) = Main.Base.include(@__MODULE__, fname)
end

# Make new names w/ a copy of the value. For functions, make a new function object.
# TODO: Currently this just makes new function objects, with methods that just call the
# corresponding method in the old module. For mocking, they would need to have the source
# code copied over as well, and modified to call the corresponding function in the new module.
deepcopy_value(destmodule, srcmodule, name, value) = Core.eval(destmodule, :($name = $(deepcopy(value)))
function deepcopy_value(destmodule, srcmodule, name, value::Function)
    @eval destmodule function $name end
    src_ms = Core.eval(srcmodule, :(methods($value).ms))
    for m in src_ms
        argnames = collect(Symbol("_$i") for i in 1:m.nargs-1)
        argsigs = collect(:($(argnames[i])::$(m.sig.parameters[i+1])) for i in 1:m.nargs-1)
        @eval destmodule $name($(argsigs...)) = $srcmodule.$name($(argnames...))
    end
end

# EWWWWWWWW, this is super hacky. This would need to be an additional C API either to allow
# setting a module's Parent, or to allow constructing a module with a parent.
setparent!(m::Module, p::Module) =
    unsafe_store!(Ptr{_Module2}(pointer_from_objref(m)),
                   _Module2(nameof(m), p), 1)
struct _Module2
    name
    parent
end
# ------------------------------------------------------------------------------------------

What do people think? Would something like this be useful and worth pursuing?
If not, is the distributed mocking approach cool, and should we investigate that further?

Thanks!

1 Like

Okay! So, I’ve gone ahead and written something to do this! :slight_smile:

Seems like it works well in my tests so far, and now we’re using it for mock-testing in my work!


Here’s the entirety of our mocking framework now that we have this package to build on:

macro mock_test(_module, setup, test)
    quote
        m2 = $DeepcopyModules.deepcopy_module($(esc(_module)))
        @eval m2 eval($(Expr(:quote, setup)))
        @eval m2 eval($(Expr(:quote, test)))
    end
end

That function is also available in the examples/mocking.jl file of the DeepcopyModules repo.

And it’s used like this:

julia> @mock_test(
           AppMain,
           begin
               using Test
               run_app_was_triggerred = fill(false)
               AppMain.run_app() = run_app_was_triggerred[] = true
           end,
           @testset "run_app" begin
               AppMain.main(["run"])
               @test run_app_was_triggerred[]
           end)
Test Summary: | Pass  Total
run_app       |    1      1
Test.DefaultTestSet("run_app", Any[], 1, false)

If others are interested, we can package up the mock-testing framework into its own package for others to use as well!

2 Likes

What is the difference between deepcopy_module(JSON) and Base.include(Module(), Base.locate_package(Base.PkgId(Base.UUID("682c06a0-de6a-54ab-a142-c8b1cf79cde6"), "JSON"))).JSON (in case the module you want to copy is a package)?

Maybe deepcopy_module(A) copies methods for functions in A defined in (say) a downstream package B as well? That would be impossible with include.

1 Like

Great work! This answers a question that I’ve been struggling with for a few days now: how to modify a CodeInfo object and turn it into a new function. Your code shows exactly how to do this.

Would you mind putting this under an open-source license, so that I can borrow a few lines from your code?

2 Likes

Oh whoops! Thanks, sorry, I didn’t realize I’d forgotten the license! :slight_smile: Just added an MIT License. :slight_smile:

1 Like

Ooh, good question!

Well, one difference of course is that deepcopy_module(M) would copy the current state of M, including the state in any global variables, methods that have been added to functions (as you note), etc.

But your suggestion is a good one as well, which I hadn’t considered! Doing the include in that temporary Module is a nice way to make a copy from scratch. :slight_smile:

Your suggestion is also safer/simpler, because you’re recreating it from scratch. It’s very likely that I’ve missed things in my implementation that will still be shallow-copies (for example I just noticed it’s only shallow-copying the struct definitions).

2 Likes

Wow, I didn’t know that it’s even possible (without modifying Julia itself)! I guess include still is a clean stupid-simple solution for “mocking” my own module that I’m testing. But I can see deepcopy_module would be very useful for mocking upstream packages.