Can PackageCompiler.jl compile just parts of packages?

General Usage
1

I’m developing a package, and it takes quite a while to compile (~30s), which is long enough to be annoying when trying to test/bugfix/retest. PackageCompiler.jl is awesome and can get rid of the compilation overhead (by creating a .so), so a test run takes <1s instead of ~30s, but that’s not helpful if I need to edit the code and re-run. However, at any one time I’ll only be working on part of the code - it would be wonderful if the other parts could be compiled into a .so, while leaving the parts that I’m working on to be recompiled on each run. Is this possible? To be concrete, I’d like to be able to pass a list of submodules or functions from my package to the create_sysimage function, and have only those be added.

2

Are you using Revise.jl already?

2 Likes
3

I’m not - I’ve seen the name a couple of times but not looked into that package yet.

Thanks! I’ll put Revise.jl on my reading list.

1 Like
4

If you’re developing a package you should definitely try out Revise the next time you open a Julia REPL.

It is so crucial that it is in my autostart and the first one I load in Pluto/Jupyter.

5

Revise.jl does look very nice. On a very quick test though, it doesn’t seem to work with PackageCompiler.jl - in the sense that if I precompile my package with PackageCompiler.create_sysimage() (even passing a script to the precompile_execution_file argument that has using Revise before using my_package) then even using the REPL and calling using Revise before using my_package, and editing my_package after doing both of those, the functions called from my_package are the ones that were compiled into the .so, not the edited ones. It’s not a big issue, as I guess the best practice is to just keep the REPL open and develop using Revise.jl, so PackageCompiler.jl is not needed during development - it would be nice to combine the benefits though!

6

One option is to use PackageCompiler for all your dependencies but not the code you’re actively working on, and then use Revise for the code you are developing.

3 Likes
7

Yeah, that seems like it might be the best option. I’m going to try out that workflow and see how it goes!

1 Like
8

I have a function to track a package that is already compiled into a sysimage:

const SYSIMAGE_TIME = time()

function track_sysimage_package(p)
    id = Base.PkgId(p)
    Revise.watch_package(id)
    pkgdata = Revise.pkgdatas[id]
    for file in Revise.srcfiles(pkgdata)
        fullfile = joinpath(Revise.basedir(pkgdata), file)
        if stat(fullfile).mtime > SYSIMAGE_TIME
            push!(Revise.revision_queue, (pkgdata, file))
        end
    end
end

Can it be improved? The speedup is huge!

https://github.com/timholy/Revise.jl/issues/685

1 Like