I am learning to use BinaryBuilder for making a binary dependency (ElTopo) available on Linux, Win and Mac. On Linux I use a following recipe to produce a shared library eltopo.so:
git clone https://github.com/tysonbrochu/eltopo
cd eltopo/eltopo3d
cat <<EOF > Makefile.local_defs
INCLUDE_PATH = -I. -I../common -I../common/newsparse -I../common/meshes -I../common/tunicate
DEPEND = g++ -D__LITTLE_ENDIAN__ -DUSE_FORTRAN_BLAS -DNO_GUI
CC = g++ -Wall -D__LITTLE_ENDIAN__ -DUSE_FORTRAN_BLAS -DNO_GUI -fPIC
RELEASE_FLAGS = -O3 -funroll-loops
DEBUG_FLAGS = -g
LINK = g++
LINK_LIBS = -lGL -lGLU -lglut -llapack -lblas
EOF
make release
make depend
g++ obj/*.o -o eltopo.so -fPIC -shared -llapack -lblas -lstdc++ -lm -I../common -I.
But that does not work inside the sandboxed environment of BinaryBuilder complaining that I don’t have LAPACK and BLAS:
/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/bin/ld: cannot find -llapack
/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/4.8.5/../../../../x86_64-linux-gnu/bin/ld: cannot find -lblas
How could I get LAPACK and BLAS inside sandboxed environment?
I think the right way would be to depend on https://github.com/JuliaLinearAlgebra/OpenBLASBuilder
IIRC the Wizard has an option for that.
1 Like
Thanks that actually worked! I replaced -lblas and -llapacwith -L./destdir/lib -lopenblas64_. But I do have another question. My build_tarbals.jl script now looks as follows
# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
using BinaryBuilder
name = "ElTopoBinary"
version = v"0.1.0"
# Collection of sources required to build ElTopoBinary
sources = [
"https://github.com/tysonbrochu/eltopo.git" =>
"14b1d7cbd45def90cfce04d381e92c4fefc5fab7",
]
# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir
cd eltopo/eltopo3d/
cat <<EOF > Makefile.local_defs
INCLUDE_PATH = -I. -I../common -I../common/newsparse -I../common/meshes -I../common/tunicate
DEPEND = g++ -D__LITTLE_ENDIAN__ -DUSE_FORTRAN_BLAS -DNO_GUI
CC = g++ -Wall -D__LITTLE_ENDIAN__ -DUSE_FORTRAN_BLAS -DNO_GUI -fPIC
RELEASE_FLAGS = -O3 -funroll-loops
DEBUG_FLAGS = -g
LINK = g++
LINK_LIBS = -lGL -lGLU -lglut -llapack -lblas
EOF
make depend
make release
cd $WORKSPACE
(cd srcdir/eltopo/eltopo3d && find . -name '*.h' -print | tar --create --files-from -) | (cd $WORKSPACE/destdir/include && tar xvfp -)
(cd srcdir/eltopo/common && find . -name '*.h' -print | tar --create --files-from -) | (cd $WORKSPACE/destdir/include && tar xvfp -)
g++ srcdir/eltopo/eltopo3d/obj/*.o -o destdir/lib/eltopo.so -fPIC -shared -L./destdir/lib -lopenblas64_ -lstdc++ -lm
"""
# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
platforms = [
Linux(:x86_64, :glibc)
#Linux(:x86_64, libc=:glibc)
# Linux(:aarch64, libc=:glibc),
# Linux(:powerpc64le, libc=:glibc),
# Linux(:x86_64, libc=:musl),
# Linux(:aarch64, libc=:musl)
]
# The products that we will ensure are always built
products(prefix) = [
LibraryProduct(prefix, "eltopo", :eltopo)
]
# Dependencies that must be installed before this package can be built
dependencies = [
"https://github.com/JuliaLinearAlgebra/OpenBLASBuilder/releases/download/v0.3.0-2/build_OpenBLAS.v0.3.0.jl"
]
# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies)
When I execute it julia build_tarballs.jl I get an error after the build script had been finished
ERROR: LoadError: KeyError: key Linux(:x86_64, libc=:glibc) not found
Stacktrace:
[1] getindex(::Dict{Platform,Tuple{String,String}}, ::Linux) at ./dict.jl:478
[2] top-level scope at /home/janiserdmanis/.julia/packages/BinaryBuilder/Y2V0i/src/AutoBuild.jl:412
in expression starting at /home/janiserdmanis/BtSync/Projects/Julia/BinaryBuilder/build/build_tarballs.jl:60
What does it mean?
@staticfloat might be able to help you here, but which version of BinaryBuilder are you one?
There recently was a change to the way platforms work BinaryBuilder and GCC multi-versioning
I believe this was a bug in BB that was just fixed. Please update your BinaryBuilder checkout and try again.
1 Like
I was on master. I will check it out again later today.
Thanks! The build script works and produces a product 
I am amazed by the result. Only binaries and my added header files without stuff of openblas. Seems that building and depending on a binary is going to become a fun thing to do :))
Hold on. Am I reading this correctly? If a package depends on BLAS and LAPACK, the recommended way is to package an additional copy of openBLAS rather than just use the BLAS and LAPACK that Julia already depends on? Why is that?
In the binary there is nothing of openBLAS except a link to a shared library libopneblas64_.so which is only present at the building stage. When tarball is built the openBLAS files are removed so there are no additional copies. I expect there is some magic with BinaryProvider which adds soft links to the libraries for the linking when one tries to install it.(But I will find out that soon.) As I understand, there is nothing which would restrict to make soft links to the same shared library or even to the system one by implementing something similar as update-alternatives from Debian.
After running some more tests with the compiled shared library which depends on openBLAS I found a following error:
julia: symbol lookup error: /home/janiserdmanis/BtSync/Projects/Julia/ElTopo.jl/deps/usr/lib/eltopo.so: undefined symbol: dsyev_
Looking up I found dsyev_ is some kind of BLAS function so I looked if the compiled library eltopo.so is linked to openBLAS library:
[janiserdmanis][~/BtSync/Projects/Julia/ElTopo.jl/deps/usr/lib] $ ldd eltopo.so
linux-vdso.so.1 (0x00007ffd109e8000)
libgtk3-nocsd.so.0 => /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0 (0x00007f7c0f096000)
libopenblas64_.so.0 => /home/janiserdmanis/BtSync/Projects/Julia/ElTopo.jl/deps/usr/lib/./libopenblas64_.so.0 (0x00007f7c0caff000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f7c0c771000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7c0c3d3000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7c0c1bb000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7c0bdca000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f7c0bbc6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7c0b9a7000)
libgfortran.so.4 => /usr/lib/x86_64-linux-gnu/libgfortran.so.4 (0x00007f7c0b5c8000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7c0f55b000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f7c0b388000)
where I see that linking seems to be fine. Then I looked up if the symbol is present in libopenblas64_.so.0:
[janiserdmanis][~/BtSync/Projects/Julia/ElTopo.jl/deps/usr/lib] $ nm -D libopenblas64_.so.0 | grep dsyev_
0000000001bdd190 T dsyev_2stage_64_
0000000001b5a330 T dsyev_64_
000000000203d4b0 T LAPACKE_dsyev_2stage64_
000000000203d620 T LAPACKE_dsyev_2stage_work64_
000000000203bce0 T LAPACKE_dsyev_work64_
which reveals that for BinaryBuilder’s openBLAS there is a suffix *64_. Looking up SundialsBuilder seems that the recommended way is to patch the code before compiling like with this one. However that patch seems to be limited to a certain symbols. Are there some universal patch for mangling BLAS and LAPACK symbols to get *64_ suffix?
If I try to link with openBLAS static object libopenblas64_.a I get a following error upon execution of problematic code:
julia: symbol lookup error: /home/janiserdmanis/BtSync/Projects/Julia/ElTopo.jl/deps/usr/lib/eltopo.so: undefined symbol: _gfortran_pow_r8_i8
EDIT: Adding -lgfortran flag fixed the issue above with static linking.
EDIT2: After even more testing I got an unexpected error at runtime:
** On entry to DSYEV Safe minimumPrecisionM parameter number 3 had an illegal value