BinaryBuilder -- unable to find shared library product

Tooling Package Management
1

Hi – I’m using the BinaryBuilder CLI to create a JLL for the liquid-dsp github repo. The build is failing during the audit as unable to find the shared library – appears to be looking in the wrong build directories? My build script is below, the CLI command, and so on. Any help would be appreciated!

using BinaryBuilder

name = "libliquid"
version = v"1.7.0"
sources = [
    GitSource("https://github.com/jgaeddert/liquid-dsp.git", "a8cc94a6f1f4386c294f5609dc2a373806cafd9c"),
    DirectorySource("./bundled"),
]

script = raw"""
cd ${WORKSPACE}/srcdir/liquid-dsp

mkdir build
cd build

cmake ..
make
make install

install_license ${WORKSPACE}/srcdir/liquid-dsp/LICENSE
"""

platforms = filter(x->contains(triplet(x), "x86_64-linux-gnu"), supported_platforms(experimental=false))

products = [
    LibraryProduct("libliquid", :libliquid),
]

dependencies = [
    Dependency("FFTW_jll"),
]

build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies, julia_compat="1.7")

Output from running BinaryBuilder via CLI :

$:~/projects/tmp2$ julia +1.7 --project=./ --color=yes build_tarballs.jl 

[ Info: Building for x86_64-linux-gnu
┌ Warning: Unable to run unprivileged containers on this system! This may be because your kernel does not support mounting overlay filesystems within user namespaces. To work around this, we will switch to using privileged containers. This requires the use of sudo. To choose this automatically, set the BINARYBUILDER_RUNNER environment variable to "privileged" before starting Julia.
└ @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/yt9V4/src/UserNSRunner.jl:97
[ Info: Running privileged container via `sudo`, may ask for your password:
 
[ Info: Found a valid dl path libfftw3.so while looking for libliquid
[ Info: Found a valid dl path libfftw3.so.3 while looking for libliquid
[ Info: Found a valid dl path libfftw3.so.3.6.10 while looking for libliquid
[ Info: Found a valid dl path libfftw3f.so while looking for libliquid
[ Info: Found a valid dl path libfftw3f.so.3 while looking for libliquid
[ Info: Found a valid dl path libfftw3f.so.3.6.10 while looking for libliquid
[ Info: Could not locate libliquid inside ["/home/XXXX/projects/tmp2/build/x86_64-linux-gnu/TgI3iTLB/x86_64-linux-gnu-libgfortran3-cxx03/destdir/lib64", "/home/XXXX/projects/tmp2/build/x86_64-linux-gnu/TgI3iTLB/x86_64-linux-gnu-libgfortran3-cxx03/destdir/lib"]
┌ Error: Built libliquid but libliquid still unsatisfied:
└ @ BinaryBuilder ~/.julia/packages/BinaryBuilder/3eUfg/src/AutoBuild.jl:987
ERROR: LoadError: Cannot continue with unsatisfied build products!
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] autobuild(dir::AbstractString, src_name::AbstractString, src_version::VersionNumber, sources::Vector{<:BinaryBuilderBase.AbstractSource}, script::AbstractString, platforms::Vector, products::Vector{<:Product}, dependencies::Vector{<:BinaryBuilderBase.AbstractDependency}; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, dont_dlopen::Bool, compression_format::String, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
   @ BinaryBuilder ~/.julia/packages/BinaryBuilder/3eUfg/src/AutoBuild.jl:992
 [3] build_tarballs(ARGS::Any, src_name::Any, src_version::Any, sources::Any, script::Any, platforms::Any, products::Any, dependencies::Any; julia_compat::String, validate_name::Bool, compression_format::String, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
   @ BinaryBuilder ~/.julia/packages/BinaryBuilder/3eUfg/src/AutoBuild.jl:380
 [4] top-level scope
   @ ~/projects/tmp2/build_tarballs.jl:35
in expression starting at /home/XXXX/projects/tmp2/build_tarballs.jl:35

Using the find command to confirm shared library does exist:

$:~/projects/tmp2$ find . -name *libliquid.so* -print

./build/x86_64-linux-gnu/TgI3iTLB/srcdir/liquid-dsp/build/libliquid.so
./build/x86_64-linux-gnu/TgI3iTLB/srcdir/liquid-dsp/build/libliquid.so.1.7.0
./build/x86_64-linux-gnu/TgI3iTLB/srcdir/liquid-dsp/build/libliquid.so.1
2

Please read the documentation

What BinaryBuilder does is to create a tarball containing all files that are found inside the ${prefix} directory at the end of the build and which don’t come from the dependencies listed in the build recipe.
Thus, what you want to do in a build script is to install the relevant files under the appropriate directories in ${prefix} (see the Automatic environment variables section): the libraries in ${libdir}, the binary executables in ${bindir}, etc…
Most packages come with a build system to automate this process (GNU Autoconf, CMake, Meson, a plain Makefile, etc…), but sometimes you may need to manually move the files as appropriate.

For cmake builds specifically see Build Tips · BinaryBuilder.jl

3

Ok, it wasn’t clear to me in reading the documents that I had to ensure the shared library product was built (or moved) into the {libdir}. Thanks for emphasizing that section. I was able to successfully compile for for the two x86_64-linux platforms which is my requirement. I ran into issues trying to compile against the rest but I’m not sufficiently motivated to address.

I’ll see if I can do a pull request to get this into Yggdrasil. Perhaps it can be template for others if they want to expand to other platforms.

For posterity and those that care, the script portion of the build_tarballs.jl listed in the original post was modified to the following:

script = raw"""
cd ${WORKSPACE}/srcdir/liquid-dsp

mkdir build
cd build

cmake ..
make
make install

#need to specify license location
install_license ${WORKSPACE}/srcdir/liquid-dsp/LICENSE

# Move libraries to ${libdir} on for linux builds
if [[ "${target}" == *linux* ]]; then
    mv libliquid.so* "${libdir}"
fi

"""
4

You should really follow the instructions in the cmake section at Build Tips · BinaryBuilder.jl. Setting the install prefix and using the correct toolchain file is rather important.

5

I’ll take a look to see if I can figure it out. If there are relevant examples you can point me towards that are similar to what I’m working on, that would be great!

6

Setting to the default BinaryBuilder cmake set of commands results in a failed build. Next, swapping my minimally successful cmake .. line with BinaryBuilder (removing the -B build flag) , e.g.,

cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} -DCMAKE_BUILD_TYPE=Release

results in a failed build, specifically an incomplete configuring with errors (same as default BinaryBuilder cmake). In fact, any combination of keeping/removing prefix, toolchain, or ready setting resulted in a failed build.

Looking at the verbose configuration summary, the successful build doesn’t find an external FFTW shared library and install path is /usr/local.

Minimally successful build

[19:48:49] – ******************* liquid-dsp Configuration Summary *******************
[19:48:49] – General:
[19:48:49] – Version : 1.7.0
[19:48:49] – System : Linux
[19:48:49] – C++ compiler : /opt/bin/x86_64-linux-gnu-libgfortran3-cxx03/c++
[19:48:49] – Release CXX flags : -O3 -DNDEBUG
[19:48:49] – Debug CXX flags : -g
[19:48:49] – Build type :
[19:48:49] –
[19:48:49] – BUILD_EXAMPLES : ON
[19:48:49] – BUILD_AUTOTESTS : ON
[19:48:49] – BUILD_BENCHMARKS : ON
[19:48:49] – BUILD_SHARED_LIBS : ON
[19:48:49] – ENABLE_SIMD : ON
[19:48:49] – FIND_SIMD : ON
[19:48:49] – BUILD_SANDBOX : OFF
[19:48:49] – BUILD_DOC : OFF
[19:48:49] – COVERAGE : OFF
[19:48:49] –
[19:48:49] – Acceleration:
[19:48:49] – NEON : No
[19:48:49] – SSE4 : Yes, flags=’ ’
[19:48:49] – AVX : Yes, flags=‘-mavx’
[19:48:49] – AVX2 : Yes, flags=‘-mavx2 -mfma -mf16c’
[19:48:49] – AVX512 : No
[19:48:49] – AltiVec : No
[19:48:49] –
[19:48:49] – External:
[19:48:49] – FFTW : No
[19:48:49] –
[19:48:49] – Install:
[19:48:49] – Install path : /usr/local
[19:48:49] –
[19:48:49] – Configuring done
[19:48:49] – Generating done
[19:48:49] – Build files have been written to: /workspace/srcdir/liquid-dsp/build
[19:48:49] —> make

As apposed to the failed build which found an external FFTW library on the host system and set the install path to /workspace/destdir. Hmmm, but a different compiler also.

Failed build

[19:59:28] – ******************* liquid-dsp Configuration Summary *******************
[19:59:28] – General:
[19:59:28] – Version : 1.7.0
[19:59:28] – System : Linux
[19:59:28] – C++ compiler : /opt/bin/x86_64-linux-gnu-libgfortran3-cxx03/x86_64-linux-gnu-g++
[19:59:28] – Release CXX flags : -O3 -DNDEBUG
[19:59:28] – Debug CXX flags : -g
[19:59:28] – Build type : Release
[19:59:28] –
[19:59:28] – BUILD_EXAMPLES : ON
[19:59:28] – BUILD_AUTOTESTS : ON
[19:59:28] – BUILD_BENCHMARKS : ON
[19:59:28] – BUILD_SHARED_LIBS : ON
[19:59:28] – ENABLE_SIMD : ON
[19:59:28] – FIND_SIMD : ON
[19:59:28] – BUILD_SANDBOX : OFF
[19:59:28] – BUILD_DOC : OFF
[19:59:28] – COVERAGE : OFF
[19:59:28] –
[19:59:28] – Acceleration:
[19:59:28] – NEON : No
[19:59:28] – SSE4 : Yes, flags=’ ’
[19:59:28] – AVX : Yes, flags=‘-mavx’
[19:59:28] – AVX2 : Yes, flags=‘-mavx2 -mfma -mf16c’
[19:59:28] – AVX512 : No
[19:59:28] – AltiVec : No
[19:59:28] –
[19:59:28] – External:
[19:59:28] – FFTW : include path=‘/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include’
[19:59:28] – : library path=‘/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/lib/libfftw3f.so’
[19:59:28] –
[19:59:28] – Install:
[19:59:28] – Install path : /workspace/destdir
[19:59:28] –
[19:59:28] – Configuring incomplete, errors occurred!
[19:59:28] See also “/workspace/srcdir/liquid-dsp/build/CMakeFiles/CMakeOutput.log”.
[19:59:28] See also “/workspace/srcdir/liquid-dsp/build/CMakeFiles/CMakeError.log”.

7

Digging at this some more – see build script below – dropping into debug mode and rerunning the parameterized cmake leads to a successful configuration and the completion of that command. It then fails on themake command due to a source repo autotest of the optional FFT library…

build script 
using BinaryBuilder

name = "LiquidDSP"
version = v"1.7.0"
sources = [ 
    GitSource("https://github.com/jgaeddert/liquid-dsp.git", "a8cc94a6f1f4386c294f5609dc2a373806cafdd
9c"),
    DirectorySource("./bundled"),
]

script = raw"""
cd ${WORKSPACE}/srcdir/liquid-dsp

##########using liquid-dsp repo instructions
#mkdir build
#cd build
#cmake ..
#make
#make install

#########default cmake for binary builder
#scenario 1
#cmake -B build -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} -DDCMAKE_BUILD_TYPE=Release
#cmake --build build --parallel ${nproc}
#cmake --install build

############hybrid
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} -DCMAKE_BBUILD_TYPE=Release
make
make install
#################################

#need to specify license location
install_license ${WORKSPACE}/srcdir/liquid-dsp/LICENSE

# Move libraries to ${libdir} on for linux builds
if [[ "${target}" == *linux* ]]; then
    mv libliquid.so* "${libdir}"
fi

"""

platforms = filter(x->contains(triplet(x), "x86_64-linux-gnu"), supported_platforms(experimental=fall
se))
#platforms = filter(x->contains(triplet(x), "linux"), supported_platforms(experimental=false))
#platforms = [HostPlatform()]

products = [
    LibraryProduct("libliquid", :libliquid),
]

dependencies = [
    Dependency("FFTW_jll"),
]

build_tarballs(ARGS, name, version, sources, script,
               platforms, products, dependencies,
               julia_compat="1.7")
make error due to optional FFT_jll 
[ 51%] Linking C executable xautotest
/usr/bin/cmake -E cmake_link_script CMakeFiles/xautotest.dir/link.txt --verbose=true
... a wall of test and then the last few lines...
/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root//usr/local/lib/libfftw3f.so: undefined reference to `memcpy@GLIBC_2.14'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/xautotest.dir/build.make:4618: xautotest] Error 1
make[2]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
make[1]: *** [CMakeFiles/Makefile2:1109: CMakeFiles/xautotest.dir/all] Error 2
make[1]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
make: *** [Makefile:166: all] Error 2

Removing the optional FFT_jll build dependency and going through process again (including re-running the parameterized cmake command) in debug mode, I can successfully manually step through make and make install.

Any insight on troubleshooting the need to rerun the parameterized cmake? Or the separate FFT reference issue?

EDIT: Added in the full build script

8

You didn’t provide the full build_tarballs.jl script, but my educated guess is that you’ll solve your problem by adding the keyword argument preferred_gcc_version=v"5" to the build_tarballs(...) function call.

9

Ok, I edited the previous post to show the entire build script. I added in your keyword argument to that build script…

build_tarballs(ARGS, name, version, sources, script,
               platforms, products, dependencies,
               julia_compat="1.7",
               preferred_gcc_version=v"5")

… and results are similar. Still have to rerun the parameterized cmake line with the source repo reporting configuration errors the first instance with or without the optional FFT_jll dependency. The subsequent make command does fail at a different (early spot) with the optional FFT_jll.

10

That sounds unlikely, GCC 5 comes with the right version of glibc.

It may be more effective to continue this in a PR in Yggdrasil.

11

Yeah, I should be more specific. It isn’t glibc error now but

libliquid.so.1.7.0: undefined reference to `_mm512_mask_reduce_add_ps'
libliquid.so.1.7.0: undefined reference to `_mm512_reduce_add_ps'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/xautotest.dir/build.make:4618: xautotest] Error 1
make[2]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
make[1]: *** [CMakeFiles/Makefile2:1109: CMakeFiles/xautotest.dir/all] Error 2
make[1]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
make: *** [Makefile:166: all] Error 2

I’ll go ahead and put a PR in Yggdrasil.

12

:slightly_smiling_face:

Ok, that’s a different error, it means the compiler doesn’t know the _mm512_mask_reduce_add_ps/_mm512_reduce_add_ps intrinsics, you’ll need to figure out in which version of GCC they were introduced. Edit: based on re PR target/80324 (_mm512_reduce_xxx type instrinsics are missing) · gcc-mirror/gcc@167a5b7 · GitHub, that’s GCC 8.

13

Awesome! Setting preferred_gcc_version=v"8" was successful. Now I’ve isolated the remaining(?) problemto that the CMAKE_TARGET_TOOLCHAIN isn’t playing nicely. If I remove that argument, I can build successfully for Linux x86_64 but nothing else. If I keep it in, I always get a configuration error regardless of target platform. I’ve attached some debug output that has various CMake Error: TRY_RUN() invoked in cross-compiling mode errors but unclear if they are fatal errors or part of the availability testing.

configuration error 
[07:56:14] 
[07:56:14] ` 
[07:56:14]  ---> cd ${WORKSPACE}/srcdir/liquid-dsp
[07:56:14]  ---> mkdir build
[07:56:14]  ---> cd build
[07:56:14]  ---> cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN}
[07:56:14] -- The C compiler identification is GNU 8.1.0
[07:56:14] -- The CXX compiler identification is GNU 8.1.0
[07:56:14] -- Detecting C compiler ABI info
[07:56:15] -- Detecting C compiler ABI info - done
[07:56:15] -- Check for working C compiler: /opt/bin/x86_64-linux-gnu-libgfortran5-cxx11/x86_64-linux-gnu-gcc - skipped
[07:56:15] -- Detecting C compile features
[07:56:15] -- Detecting C compile features - done
[07:56:15] -- Detecting CXX compiler ABI info
[07:56:15] -- Detecting CXX compiler ABI info - done
[07:56:15] -- Check for working CXX compiler: /opt/bin/x86_64-linux-gnu-libgfortran5-cxx11/x86_64-linux-gnu-g++ - skipped
[07:56:15] -- Detecting CXX compile features
[07:56:15] -- Detecting CXX compile features - done
[07:56:15] -- Preparing  Test C_HAS_SSE4_1: ' '
[07:56:15] -- Performing Test C_HAS_SSE4_1
[07:56:16] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:16]    C_HAS_SSE4_1_EXITCODE (advanced)
[07:56:16]    C_HAS_SSE4_1_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:16] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:16] -- Performing Test C_HAS_SSE4_1 - Failed
[07:56:16] -- Skipping   Test C_HAS_SSE4_2: -msse4.2
[07:56:16] -- Skipping   Test C_HAS_SSE4_2: /arch:SSE
[07:56:16] -- Preparing  Test C_HAS_AVX_1: ' '
[07:56:16] -- Performing Test C_HAS_AVX_1
[07:56:16] -- Performing Test C_HAS_AVX_1 - Failed
[07:56:16] -- Preparing  Test C_HAS_AVX_2: '-mavx'
[07:56:16] -- Performing Test C_HAS_AVX_2
[07:56:16] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:16]    C_HAS_AVX_2_EXITCODE (advanced)
[07:56:16]    C_HAS_AVX_2_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:16] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:16] -- Performing Test C_HAS_AVX_2 - Failed
[07:56:16] -- Skipping   Test C_HAS_AVX_3: /arch:AVX
[07:56:16] -- Preparing  Test C_HAS_AVX2_1: ' '
[07:56:16] -- Performing Test C_HAS_AVX2_1
[07:56:17] -- Performing Test C_HAS_AVX2_1 - Failed
[07:56:17] -- Preparing  Test C_HAS_AVX2_2: '-mavx2 -mfma -mf16c'
[07:56:17] -- Performing Test C_HAS_AVX2_2
[07:56:17] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:17]    C_HAS_AVX2_2_EXITCODE (advanced)
[07:56:17]    C_HAS_AVX2_2_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:17] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:17] -- Performing Test C_HAS_AVX2_2 - Failed
[07:56:17] -- Skipping   Test C_HAS_AVX2_3: /arch:AVX2
[07:56:17] -- Preparing  Test C_HAS_AVX512_1: ' '
[07:56:17] -- Performing Test C_HAS_AVX512_1
[07:56:18] -- Performing Test C_HAS_AVX512_1 - Failed
[07:56:18] -- Preparing  Test C_HAS_AVX512_2: '-mavx512f -mavx512dq -mavx512vl -mavx512bw -mfma'
[07:56:18] -- Performing Test C_HAS_AVX512_2
[07:56:18] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:18]    C_HAS_AVX512_2_EXITCODE (advanced)
[07:56:18]    C_HAS_AVX512_2_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:18] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:18] -- Performing Test C_HAS_AVX512_2 - Failed
[07:56:18] -- Skipping   Test C_HAS_AVX512_3: /arch:AVX512
[07:56:18] -- Preparing  Test C_HAS_NEON_1: ' '
[07:56:18] -- Performing Test C_HAS_NEON_1
[07:56:18] -- Performing Test C_HAS_NEON_1 - Failed
[07:56:18] -- Preparing  Test C_HAS_NEON_2: '-ffast-math'
[07:56:18] -- Performing Test C_HAS_NEON_2
[07:56:18] -- Performing Test C_HAS_NEON_2 - Failed
[07:56:18] -- Preparing  Test C_HAS_NEON_3: '/arch:armv8.0'
[07:56:18] -- Performing Test C_HAS_NEON_3
[07:56:18] -- Performing Test C_HAS_NEON_3 - Failed
[07:56:18] -- Preparing  Test C_HAS_NEON_4: '/arch:armv9.0'
[07:56:18] -- Performing Test C_HAS_NEON_4
[07:56:18] -- Performing Test C_HAS_NEON_4 - Failed
[07:56:18] -- Preparing  Test C_HAS_ALTIVEC_1: ' '
[07:56:18] -- Performing Test C_HAS_ALTIVEC_1
[07:56:18] -- Performing Test C_HAS_ALTIVEC_1 - Failed
[07:56:18] -- Preparing  Test C_HAS_ALTIVEC_2: '-fno-common -faltivec'
[07:56:18] -- Performing Test C_HAS_ALTIVEC_2
[07:56:18] -- Performing Test C_HAS_ALTIVEC_2 - Failed
[07:56:18] -- Preparing  Test C_HAS_ALTIVEC_3: '/arch:altivec'
[07:56:18] -- Performing Test C_HAS_ALTIVEC_3
[07:56:19] -- Performing Test C_HAS_ALTIVEC_3 - Failed
[07:56:19] -- Preparing  Test CXX_HAS_SSE4_1: ' '
[07:56:19] -- Performing Test CXX_HAS_SSE4_1
[07:56:19] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:19]    CXX_HAS_SSE4_1_EXITCODE (advanced)
[07:56:19]    CXX_HAS_SSE4_1_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:19] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:19] -- Performing Test CXX_HAS_SSE4_1 - Failed
[07:56:19] -- Skipping   Test CXX_HAS_SSE4_2: -msse4.2
[07:56:19] -- Skipping   Test CXX_HAS_SSE4_2: /arch:SSE
[07:56:19] -- Preparing  Test CXX_HAS_AVX_1: ' '
[07:56:19] -- Performing Test CXX_HAS_AVX_1
[07:56:19] -- Performing Test CXX_HAS_AVX_1 - Failed
[07:56:19] -- Preparing  Test CXX_HAS_AVX_2: '-mavx'
[07:56:19] -- Performing Test CXX_HAS_AVX_2
[07:56:20] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:20]    CXX_HAS_AVX_2_EXITCODE (advanced)
[07:56:20]    CXX_HAS_AVX_2_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:20] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:20] -- Performing Test CXX_HAS_AVX_2 - Failed
[07:56:20] -- Skipping   Test CXX_HAS_AVX_3: /arch:AVX
[07:56:20] -- Preparing  Test CXX_HAS_AVX2_1: ' '
[07:56:20] -- Performing Test CXX_HAS_AVX2_1
[07:56:20] -- Performing Test CXX_HAS_AVX2_1 - Failed
[07:56:20] -- Preparing  Test CXX_HAS_AVX2_2: '-mavx2 -mfma -mf16c'
[07:56:20] -- Performing Test CXX_HAS_AVX2_2
[07:56:21] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:21]    CXX_HAS_AVX2_2_EXITCODE (advanced)
[07:56:21]    CXX_HAS_AVX2_2_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:21] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:21] -- Performing Test CXX_HAS_AVX2_2 - Failed
[07:56:21] -- Skipping   Test CXX_HAS_AVX2_3: /arch:AVX2
[07:56:21] -- Preparing  Test CXX_HAS_AVX512_1: ' '
[07:56:21] -- Performing Test CXX_HAS_AVX512_1
[07:56:21] -- Performing Test CXX_HAS_AVX512_1 - Failed
[07:56:21] -- Preparing  Test CXX_HAS_AVX512_2: '-mavx512f -mavx512dq -mavx512vl -mavx512bw -mfma'
[07:56:21] -- Performing Test CXX_HAS_AVX512_2
[07:56:21] CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
[07:56:21]    CXX_HAS_AVX512_2_EXITCODE (advanced)
[07:56:21]    CXX_HAS_AVX512_2_EXITCODE__TRYRUN_OUTPUT (advanced)
[07:56:21] For details see /workspace/srcdir/liquid-dsp/build/TryRunResults.cmake
[07:56:21] -- Performing Test CXX_HAS_AVX512_2 - Failed
[07:56:21] -- Skipping   Test CXX_HAS_AVX512_3: /arch:AVX512
[07:56:21] -- Preparing  Test CXX_HAS_NEON_1: ' '
[07:56:21] -- Performing Test CXX_HAS_NEON_1
[07:56:21] -- Performing Test CXX_HAS_NEON_1 - Failed
[07:56:21] -- Preparing  Test CXX_HAS_NEON_2: '-ffast-math'
[07:56:21] -- Performing Test CXX_HAS_NEON_2
[07:56:22] -- Performing Test CXX_HAS_NEON_2 - Failed
[07:56:22] -- Preparing  Test CXX_HAS_NEON_3: '/arch:armv8.0'
[07:56:22] -- Performing Test CXX_HAS_NEON_3
[07:56:22] -- Performing Test CXX_HAS_NEON_3 - Failed
[07:56:22] -- Preparing  Test CXX_HAS_NEON_4: '/arch:armv9.0'
[07:56:22] -- Performing Test CXX_HAS_NEON_4
[07:56:22] -- Performing Test CXX_HAS_NEON_4 - Failed
[07:56:22] -- Preparing  Test CXX_HAS_ALTIVEC_1: ' '
[07:56:22] -- Performing Test CXX_HAS_ALTIVEC_1
[07:56:22] -- Performing Test CXX_HAS_ALTIVEC_1 - Failed
[07:56:22] -- Preparing  Test CXX_HAS_ALTIVEC_2: '-fno-common -faltivec'
[07:56:22] -- Performing Test CXX_HAS_ALTIVEC_2
[07:56:22] -- Performing Test CXX_HAS_ALTIVEC_2 - Failed
[07:56:22] -- Preparing  Test CXX_HAS_ALTIVEC_3: '/arch:altivec'
[07:56:22] -- Performing Test CXX_HAS_ALTIVEC_3
[07:56:22] -- Performing Test CXX_HAS_ALTIVEC_3 - Failed
[07:56:22] -- Found fftw3f: /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/lib/libfftw3f.so  
[07:56:22] -- Looking for sys/types.h
[07:56:22] -- Looking for sys/types.h - found
[07:56:22] -- Looking for stdint.h
[07:56:23] -- Looking for stdint.h - found
[07:56:23] -- Looking for stddef.h
[07:56:23] -- Looking for stddef.h - found
[07:56:23] -- Check size of int
[07:56:23] -- Check size of int - done
[07:56:24] -- 
[07:56:24] -- ******************* liquid-dsp Configuration Summary *******************
[07:56:24] -- General:
[07:56:24] --   Version           :   1.7.0
[07:56:24] --   System            :   Linux
[07:56:24] --   C++ compiler      :   /opt/bin/x86_64-linux-gnu-libgfortran5-cxx11/x86_64-linux-gnu-g++
[07:56:24] --   Release CXX flags :   -O3 -DNDEBUG
[07:56:24] --   Debug CXX flags   :   -g
[07:56:24] --   Build type        :   
[07:56:24] -- 
[07:56:24] --   BUILD_EXAMPLES    :   ON
[07:56:24] --   BUILD_AUTOTESTS   :   ON
[07:56:24] --   BUILD_BENCHMARKS  :   ON
[07:56:24] --   BUILD_SHARED_LIBS :   ON
[07:56:24] --   ENABLE_SIMD       :   ON
[07:56:24] --   FIND_SIMD         :   ON
[07:56:24] --   BUILD_SANDBOX     :   OFF
[07:56:24] --   BUILD_DOC         :   OFF
[07:56:24] --   COVERAGE          :   OFF
[07:56:24] -- 
[07:56:24] -- Acceleration:
[07:56:24] --   NEON              :   No
[07:56:24] --   SSE4              :   Yes, flags=' '
[07:56:24] --   AVX               :   Yes, flags='-mavx'
[07:56:24] --   AVX2              :   Yes, flags='-mavx2 -mfma -mf16c'
[07:56:24] --   AVX512            :   Yes, flags='-mavx2 -mfma -mf16c'
[07:56:24] --   AltiVec           :   No
[07:56:24] -- 
[07:56:24] -- External:
[07:56:24] --   FFTW              :   include path='/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include'
[07:56:24] --                     :   library path='/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/lib/libfftw3f.so'
[07:56:24] -- 
[07:56:24] -- Install:
[07:56:24] --   Install path      :   /workspace/destdir
[07:56:24] -- 
[07:56:24] -- Configuring incomplete, errors occurred!
[07:56:24] See also "/workspace/srcdir/liquid-dsp/build/CMakeFiles/CMakeOutput.log".
[07:56:24] See also "/workspace/srcdir/liquid-dsp/build/CMakeFiles/CMakeError.log".
[07:56:24]  ---> cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN}
[07:56:24]  ---> cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN}
[07:56:24] Previous command exited with 1
[07:56:24] Child Process exited, exit code 1
┌ Warning: Build failed, the following log files were generated:
│   - ${WORKSPACE}/srcdir/liquid-dsp/build/CMakeFiles/CMakeError.log
│   - ${WORKSPACE}/srcdir/liquid-dsp/build/CMakeFiles/CMakeOutput.log
│
14

We can’t run foreign executables. That’s fundamentally not doable. That’s really an upstream limitation if they require TRY_RUN during the build, pretty much none of all the over 1 thousands packages we have does that (also because it would just not work).

15

Looks like others had the same issue: Cross compiling from Linux to Windows using mingw-w64 is extremely difficult via cmake · Issue #403 · jgaeddert/liquid-dsp · GitHub

16

Ah, bummer. Ok, thanks for your help. I’ve at least have a local jll out of this which is helpful. I’ll poke around on the upstream package to see if/what I can do.

17

I’ve partially sidestepped the problem by updating the build such that it only runs executable for supported platforms. For remaining Linux platforms, SIMD is disabled along with associated testing and such which leads to successful builds. However, I’m still having issues with Windows and Apple even after disabling the executable testing.

EDIT: The build script below is the successful build script for all Linux, i.e., it disables the Windows and Apple platforms.

build_tarballs.jl 
using BinaryBuilder

name = "LiquidDSP"
version = v"1.7.0"
sources = [
    GitSource("https://github.com/jgaeddert/liquid-dsp.git", "a8cc94a6f1f4386c294f5609dc2a373806cafd9c"),
    DirectorySource("./bundled"),
]

script = raw"""
cd ${WORKSPACE}/srcdir/liquid-dsp

mkdir build
cd build 

#for i686 and x68_64 linux, can use native binarybuilder toolchain w/ simd
#enabled and test w/ executable.  for other linux, use toolchain but disable
#simd and any/all testing.  current linker error with windows and apple

if [[ "${target}" == *i686-linux* ]] || [[ "${target}" == *x86_64-linux* ]]; then
    cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_BUILD_TYPE=Release 
else
    cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_BUILD_TYPE=Release \
             -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
             -DFIND_SIMD=OFF -DENABLE_SIMD=OFF \
             -DBUILD_EXAMPLES=OFF -DBUILD_AUTOTESTS=OFF -DBUILD_BENCHMARKS=OFF
fi

make -j${nproc}
make install

#need to specify license location
install_license ${WORKSPACE}/srcdir/liquid-dsp/LICENSE

"""

#windows and apple excluded for time being
platforms = supported_platforms( exclude=x->(Sys.isapple(x) || Sys.iswindows(x)) )

products = [
    LibraryProduct("libliquid", :libliquid),
]

dependencies = [
    Dependency("FFTW_jll"),
]

#requires gcc v8
build_tarballs(ARGS, name, version, sources, script,
               platforms, products, dependencies,
               julia_compat="1.7",
               preferred_gcc_version=v"8")

Windows appears to be a linker error in ld: cannot find -lc

Windows error details 
...
[17:35:23] /opt/bin/i686-w64-mingw32-libgfortran5-cxx11/i686-w64-mingw32-gcc --sysroot=/opt/i686-w64-mingw32/i686-w64-mingw32/sys-root/ -O3 -DNDEBUG -shared -o libliquid.dll -Wl,--out-implib,libliquid.dll.a -Wl,--major-image-version,1,--minor-image-version,7 -Wl,--whole-archive CMakeFiles/liquid.dir/objects.a -Wl,--no-whole-archive @CMakeFiles/liquid.dir/linklibs.rsp
[17:35:23] /opt/i686-w64-mingw32/bin/../lib/gcc/i686-w64-mingw32/8.1.0/../../../../i686-w64-mingw32/bin/ld: cannot find -lc
[17:35:23] collect2: error: ld returned 1 exit status
[17:35:23] make[2]: *** [CMakeFiles/liquid.dir/build.make:406: libliquid.dll] Error 1
[17:35:23] make[2]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
[17:35:23] make[1]: *** [CMakeFiles/Makefile2:433: CMakeFiles/liquid.dir/all] Error 2
[17:35:23] make[1]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
[17:35:23] make: *** [Makefile:166: all] Error 2
[17:35:23]  ---> make -j${nproc}
[17:35:23]  ---> make -j${nproc}
[17:35:23] Previous command exited with 2
[17:35:23] Child Process exited, exit code 2

Apple error appears to be a compiler issue ld64.lld: error: undefined symbol: __divsc3

Apple error details 
[21:59:55] ld64.lld: error: undefined symbol: __divsc3
[21:59:55] >>> referenced by CMakeFiles/equalization.dir/src/equalization/src/equalizer_cccf.c.o:(symbol eqrls_cccf_step+0x1b0)
[21:59:55] >>> referenced by CMakeFiles/filter.dir/src/filter/src/cheby2.c.o:(symbol cheby2_azpkf+0x2a8)
[21:59:55] >>> referenced by CMakeFiles/filter.dir/src/filter/src/cheby2.c.o:(symbol cheby2_azpkf+0x1f4)
[21:59:55] >>> referenced 71 more times
[21:59:55] 
[21:59:55] ld64.lld: error: undefined symbol: __divdc3
[21:59:55] >>> referenced by CMakeFiles/filter.dir/src/filter/src/iirdes.c.o:(symbol bilinear_zpkf+0xf0)
[21:59:55] >>> referenced by CMakeFiles/filter.dir/src/filter/src/iirdes.c.o:(symbol bilinear_zpkf+0xbc)
[21:59:55] >>> referenced by CMakeFiles/filter.dir/src/filter/src/iirdes.c.o:(symbol bilinear_zpkf+0x7c)
[21:59:55] >>> referenced 23 more times
[21:59:55] clang: error: linker command failed with exit code 1 (use -v to see invocation)
[21:59:55] make[2]: *** [CMakeFiles/liquid.dir/build.make:403: libliquid.1.7.0.dylib] Error 1
[21:59:55] make[2]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
[21:59:55] make[1]: *** [CMakeFiles/Makefile2:433: CMakeFiles/liquid.dir/all] Error 2
[21:59:55] make[1]: Leaving directory '/workspace/srcdir/liquid-dsp/build'
[21:59:55] make: *** [Makefile:166: all] Error 2
[21:59:55]  ---> make -j${nproc}
[21:59:55]  ---> make -j${nproc}
[21:59:55] Previous command exited with 2
[21:59:55] Child Process exited, exit code 2