TravisCI build with Python deps


#1

Hello,

not sure if it’s the right place for such a question. Feel free to say so if I should ask elsewhere!

In my package (ClimateTools.jl) I use some Python dependencies that are used mostly for mapping purpose. Right now, one of the dependency is somewhat tricky to install (basemap does not install on Python 3.6 for instance).

I’m wondering how I could set up TravisCI to

  1. create a virtual env, installing the proper versions of the packages and a specific python version
  2. in Julia, install PyCall, set-up ENV["PYTHON"]="virtualenv python"; Pkg.build("PyCall")

Step #2 is OK on my side, I already do it in my .travis file, but I can get the syntax right for the first step.

Any hint would be really appreciated.

Cheers!


#2

If you put setups in .travis.yml, you’ll have trouble with CIBot when registering the package (see The current METADATA release process). Maybe put something like this in your deps/build.jl (untested):

using Pkg

if lowercase(get(ENV, "CI", "false")) == "true"
    let basepython = get(ENV, "PYTHON", "python")
        envpath = joinpath(@__DIR__, "env")
        run(`virtualenv --python=$basepython $envpath`)

        if Sys.iswindows()
            python = joinpath(envpath, "Scripts", "python.exe")
        else
            python = joinpath(envpath, "bin", "python")
        end
        run(`$python -m pip install basemap`)

        ENV["PYTHON"] = python
        Pkg.build("PyCall")
    end
end

It’s a bit ugly to build PyCall again in your build process, but you can’t avoid it if you want to change Python path.


#3

Thanks!

I guess going the deps/build.jl approach is something that might be more robust.

I’ll build on your suggestion and come back with the results!


#4

FYI I made a PR to add virtualenv support https://github.com/JuliaPy/PyCall.jl/pull/578. I think it’d make setting up test-specific Python environment easier and reproducible. In particular, you can skip Pkg.build("PyCall").


#5

Very interesting! I also think this is much more reproducible if we can set-up virtualenv. Will keep an eye open on this PR.

Still haven’t had time to test your solution.

Cheers!