My ImageQuilting.jl package has a big chunk of uncovered code because I am not sure what dependencies I should install on Travis CI before testing its GPU features. After a lot of effort refactoring and playing around with optional dependencies, I managed to have a working test suite on my local machine that includes extra tests whenever OpenCL.jl and CLFFT.jl are installed.
I am opening this thread to ask you for your help in designing a minimal .travis.yml that takes care of setting up the environment for OpenCL.jl and CLFFT.jl. Something along these lines:
language: julia
os:
- linux
- osx
before_install:
- if [ $TRAVIS_OS_NAME = "linux" ]; then
sudo apt-get -qq update
sudo apt-get install pocl
... # what else?
fi;
- if [ $TRAVIS_OS_NAME = "osx" ]; then
# Is POCL installed by default in OSX?
# what else?
fi;
script:
- if [[ -a .git/shallow ]]; then git fetch --unshallow; fi
- julia -e 'Pkg.add("OpenCL"); Pkg.add("CLFFT")'
- julia --check-bounds=yes -e 'Pkg.clone(pwd()); Pkg.build("ImageQuilting"); Pkg.test("ImageQuilting"; coverage=true)'
after_sucess:
- # etc
I tried to borrow code from .travis.yml files in those packages, but they install many other things that I am not sure are necessary for my test goals. Can we build a solution together for future reference?
On OSX you don’t need to install OpenCL since Apple is providing a implementation.
Now as you have experienced under Linux on Travis the situation is a bit more complicated… There are at least three OpenCL CPU implementations that we can use.
The first two are a pain to install on Travis and POCL is good enough for testing purposes. All three implementations vary in small ways and for OpenCL.jl we have been seeing differences in correctness and locally I try to test against as many implementations as possible. The main question is what version of POCL is available on Travis, it needs to be fairly recent in order to work well with OpenCL.jl*
If you want to use packages like CLFFT.jl you will have to install their dependencies manually, or add deps/build.jl scripts to them. The latter would be beneficial in general.
I don’t have a good general solution for this yet, but the steps we could pursue is:
Add a build script for necessary dependencies to all CL* packages
Provide a script in OpenCL.jl that sets up a compatible version of POCL on Linux
Oh, that is true, got confused with the names, fftw3 and clFFT are different things. Please let me know if you have plans to work on the deps/build.jl for OpenCL.jl, you are much more familiar with it than I am.
@vchuravy I am reading the BinDeps specification to try write a build.jl script for OpenCL.jl. How should we choose a specific driver implementation for the user when multiple implementations are available (e.g. beignet, intel, POCL)? Can we specify our preferences and let the user customize it at runtime if he needs a specific library?
I need some sort of skeleton to start adding build steps gradually:
using BinDeps
libopencl = library_dependency("libopencl", aliases=[""], os=...)
# there are multiple providers out there even for a single OS (e.g. Arch Linux)
provides(Pacman, "opencl-nvidia", libopencl)
# Beignet is available in AUR
# POCL is yet another choice
if is_apple()
# Do we need to do anything?
end
if is_windows()
# I am afraid of this one
end
The problem is that what opencl provider to use is depended on the hardware used. I think what we can reasonably do is to make sure that we pick up any existing installation of a hardware specific library and use that and if we don’t find any print a big warning and then install POCL.
I understand @vchuravy, this is definitely part of the problem. Do you have a suggestion for a build.jl skeleton? I just need pseudocode like:
if NVIDIA
# dependencies = A, B, C
end
if AMD
# dependencies = D, E, F
end
if Intel
# dependencies = G, H, I
end
# find providers and fallback to POCL if none is found
provider(...)
We could actually prompt the user to select between different drivers when they are available for a specific OS and Graphics card. But we need to start somewhere simple…
If I remember BinDeps correctly, we should be able to do:
using BinDeps
libopencl = library_dependency("libopencl", aliases=[""], os=...)
# All these packages provide a version of OpenCL (and AFAIK this is a or)
provides(Pacman,["opencl-nvidia","beignet", "opencl-amd", "opencl-catalyst", "pocl"],[libopencl])
# Do this for apt-get and YUM
....
# Sources...
# add a Source build step that does cmake and make following http://portablecl.org/docs/html/install.html#configure-build
# A bonus would be to use the LLVM that comes with Julia
For apple we should need to do anything since they ship OpenCL and I wouldn’t bother with Windows for now (until somebody complains loudly).
@vchuravy what about always build POCL from source, use it as the default and let the user pick something else at runtime with a OpenCL.jl configurable variable like OPENCL_DRIVER?