# TravisCI build with Python deps

**URL:** <https://discourse.julialang.org/t/travisci-build-with-python-deps/17289>\
**Category:** Internals & Design\
**Created:** [November 8, 2018, 2:55am UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289 "2018-11-08T02:55:37Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Balinus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/balinus/32/243_2.png) [@Balinus](https://discourse.julialang.org/u/Balinus)\
**Post date:** [November 8, 2018, 2:55am UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/1 "2018-11-08T02:55:38Z")

</div>

Hello,

not sure if it’s the right place for such a question. Feel free to say so if I should ask elsewhere!

In my package ([ClimateTools.jl](https://github.com/Balinus/ClimateTools.jl)) I use some Python dependencies that are used mostly for mapping purpose. Right now, one of the dependency is somewhat tricky to install (basemap does not install on Python 3.6 for instance).

I’m wondering how I could set up TravisCI to

1. create a virtual env, installing the proper versions of the packages and a specific python version
2. in Julia, install PyCall, set-up `ENV["PYTHON"]="virtualenv python"; Pkg.build("PyCall")`

Step #2 is OK on my side, I already do it in my .travis file, but I can get the syntax right for the first step.

Any hint would be really appreciated.

Cheers!

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [November 8, 2018, 3:18am UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/2 "2018-11-08T03:18:07Z")

</div>

If you put setups in `.travis.yml`, you’ll have trouble with CIBot when registering the package (see [The current METADATA release process](https://discourse.julialang.org/t/the-current-metadata-release-process/16672)). Maybe put something like this in your `deps/build.jl` (untested):

```julia
using Pkg

if lowercase(get(ENV, "CI", "false")) == "true"
    let basepython = get(ENV, "PYTHON", "python")
        envpath = joinpath(@ __DIR__ , "env")
        run(`virtualenv --python=$basepython $envpath`)

        if Sys.iswindows()
            python = joinpath(envpath, "Scripts", "python.exe")
        else
            python = joinpath(envpath, "bin", "python")
        end
        run(`$python -m pip install basemap`)

        ENV["PYTHON"] = python
        Pkg.build("PyCall")
    end
end

```

It’s a bit ugly to build PyCall again in your build process, but you can’t avoid it if you want to change Python path.

---

<div class="post-metadata">

**Author:** ![Balinus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/balinus/32/243_2.png) [@Balinus](https://discourse.julialang.org/u/Balinus)\
**Post date:** [November 8, 2018, 3:56pm UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/3 "2018-11-08T15:56:41Z")

</div>

Thanks!

I guess going the `deps/build.jl` approach is something that might be more robust.

I’ll build on your suggestion and come back with the results!

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [November 9, 2018, 2:13am UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/4 "2018-11-09T02:13:55Z")

</div>

FYI I made a PR to add virtualenv support [Support virtual environment by tkf · Pull Request #578 · JuliaPy/PyCall.jl · GitHub](https://github.com/JuliaPy/PyCall.jl/pull/578). I think it’d make setting up test-specific Python environment easier and reproducible. In particular, you can skip `Pkg.build("PyCall")`.

---

<div class="post-metadata">

**Author:** ![Balinus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/balinus/32/243_2.png) [@Balinus](https://discourse.julialang.org/u/Balinus)\
**Post date:** [November 9, 2018, 2:11pm UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/5 "2018-11-09T14:11:28Z")

</div>

Very interesting! I also think this is much more reproducible if we can set-up virtualenv. Will keep an eye open on this PR.

Still haven’t had time to test your solution.

Cheers!

---

<div class="post-metadata">

**Author:** ![Balinus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/balinus/32/243_2.png) [@Balinus](https://discourse.julialang.org/u/Balinus)\
**Post date:** [November 23, 2018, 10:51pm UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/6 "2018-11-23T22:51:42Z")

</div>

For reference, here’s what I did. It works on TravisCI but not JuliaCI though. I guess it’s because the `if lowercase(get(ENV, "CI", "false")) == "true"` evaluate to `false` on JuliaCI?

```julia
using Pkg

if lowercase(get(ENV, "CI", "false")) == "true"

    let basepython = get(ENV, "PYTHON", "python2")
        envpath = joinpath(@ __DIR__ , "env")
        run(`virtualenv --python=$basepython $envpath`)

        if Sys.iswindows()
            python = joinpath(envpath, "Scripts", "python.exe")
        else
            python = joinpath(envpath, "bin", "python2")
        end

        run(`$python -m pip install numpy`)
        run(`$python -m pip install scipy`)
        run(`$python -m pip install matplotlib`)
        run(`$python -m pip install https://github.com/matplotlib/basemap/archive/v1.0.7rel.tar.gz`)
        run(`$python -m pip install git+https://github.com/matplotlib/cmocean`)
        run(`$python -m pip install netcdf4`)

        ENV["PYTHON"] = python
        Pkg.build("PyCall")
    end
end

```

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [November 23, 2018, 11:48pm UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/7 "2018-11-23T23:48:04Z")

</div>

Assuming you are talking about the build here [Tag ClimateTools.jl v0.10.0 by attobot · Pull Request #19702 · JuliaLang/METADATA.jl · GitHub](https://github.com/JuliaLang/METADATA.jl/pull/19702), this is because JuliaCIBot does not have `virtualenv` program:

```julia
┌ Error: Error building `ClimateTools`: 
│ ERROR: LoadError: IOError: could not spawn `virtualenv --python=python2 /home/jrun/ClimateTools/.julia/packages/ClimateTools/KQv9Y/deps/env`: no such file or directory (ENOENT)

```

Maybe you can run `pip install --user virtualenv` first. Or alternatively just use `pip` in user mdoe, i.e., `run(`$basepython -m pip --user install numpy`)` etc.

---

<div class="post-metadata">

**Author:** ![Balinus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/balinus/32/243_2.png) [@Balinus](https://discourse.julialang.org/u/Balinus)\
**Post date:** [November 24, 2018, 4:16am UTC](https://discourse.julialang.org/t/travisci-build-with-python-deps/17289/8 "2018-11-24T04:16:17Z")

</div>

Thanks! Will try that asap. 🙂