Ccall c++ sort vector of String

Calling C++ to sort a vector of strings is more complicated than it seems. The issue is that C++ vectors and Julia vectors are different types, and so are C++ strings and Julia strings. You first need to decide whether you want to use C++ vectors or Julia vectors, C++ strings or Julia strings, or whether you want to convert them as you call C++. Converting needs to copy all data and is thus a bit slower, and if you pass the converted vector back to Julia you get a new vector, you didn’t sort the original vector. You can use C++ vectors and strings from Julia (that’s straightforward). You can also use Julia vector or strings from C++, but that’s more complicate since these new types are not std::vector or std::string, so they won’t have any of the functionality you expect.

Another complication comes form the fact that C++ uses manual memory management, whereas Julia is garbage collected. For any object you handle in C++, you need to decide who “owns” the object, i.e. who is responsible for destroy it when it is no longer needed. A lot of C++'s low-level complexity shines through here. You can use std::shared_ptr<std::vector<...>> or std::shared_ptr<std::string> here to simplify this, if your library allows you to do that.

If you want to use a large, existing C++ library, then I assume you’d be using C++ vectors and C++ strings, and use these from Julia. That’s straightforward; they type CxxVector{T} is a subtype of AbstractVector, and CxxString is a subtype of AbstractString, so Julia will just handle these types “as usual”.

CxxWrap has a nice story for this. It’s a bit more complex that one would like, but there is a good reason for this. I am not aware of a tutorial for what you are trying to do.

If you want to make something really simple work, then sort a C array of C ints instead, i.e. call the equivalent of

int arr[10] = { ... };
std::sort(&arr[0], &arr[10], std::less<int>());

from Julia.

-erik

7 Likes

I haven’t tried using clang on Windows yet, but it looks to me like you are not linking to libjulia and libcxxwrap-julia. Even if you link in those, you probably need to compile libcxxwrap-julia from source. I recommend using CMake to set up a project similar to the one here:

That way CMake should set up all link instructions correctly.

3 Likes

6 posts were split to a new topic: Windows development without admin privileges

Hi @barche, thank you , i have tried to use your cxxwrap-template. Here are the steps I did:

  • download portable version of VS Code
    -install the cmake extension
    -downloaded cmake 3.22 portable and added it to PATH
    -I cloned your cxxwrap-template repo with git on my local drive
    -I have opened the folder in VS Code
    -I have opened the file called CMakeLists.txt
    -I have run the file using the button at the bottom of VS code (like you describe on github)
    -I got an error that the variable JlCxx_DIR was not set, so I did set it to C:\Users\gitboy\.julia\artifacts\...\lib\cmake\JlCxx and re-run

I got the following message:

Cmake Warning at CMakeLists.txt:8 (find_package):
Could not find a configuration file for package "JlCxx" that is compatible with requested version "".
The following configuration files were considered but not accepted:
C:\Users\gitboy\.julia\artifacts\...\lib\cmake\JlCxx\JlCxxConfig.cmake, version: 0.9.0 (64bit)

-- Configuration incomplete, errors occured!
CMake Error at CMakeLists.txt:9 (get_target_property):
get_target_property() called with non-existent target "JlCxx::cxxwrap_julia".

CMake Error at CMakeLists.txt:10 (get_file_name_component):
get_file_name_component called with incorrect number of arguments

I am not sure to understand what is missing, would you have detailed steps on how I can make this work? Thank you

In the workspace settings of cxxwrap-template you should add something like this:

{
    "cmake.configureSettings": {
        "JlCxx_DIR": "C:/Users/user/source/repos/libcxxwrap-julia/build",
        "Julia_EXECUTABLE": "C:/Users/user/AppData/Local/Programs/Julia-1.7.1/bin/julia.exe"
    }
}

Here, C:/Users/user/source/repos/libcxxwrap-julia/build is the build directory of the libcxxwrap that you compiled with the same compiler as you are using to develop the wrapper.

Please note that this topic is closed, we should continue in Windows development without admin privileges (@stevengj you may want to move this post and the one above it, thanks :slight_smile: )

2 Likes

Thank you @barche for the details. I’ll continue posting here until they move your post to the other thread so people who are looking for help can follow the discussion.
I did what you said and it seemed to have work as I got the following messages:

[cmake] -- Configuration done
[cmake] -- Generating done
[cmake] -- Build files have been written to: D:/cxxwrap-template/build

But I don’t see any .dll or library being created that I can use with @wrapmodule.

Does your CMakeLists.txt have an install target? By default, those binaries should be located in $CMAKE_INSTALL_PREFIX/bin.

This just completed the build file generation phase, you still need to compile by hitting the “Build” button at the bottom or executing the VScode command CMake: Build. The compiled files should then appear in a subdir build

2 Likes

CMakeLists.txt files that I use exactly the same as the one in the git repo of libcxxwrap-julia and cxxwrap-template.
Following instruction of @barche , I can see now the dll in the folder D:\libcxxwrap-julia\build\bin
Then I tried tried to build the dll for cxxwrap-template and got a lot warning messages like that one:

D:\libbcxxwrap-julia\include\jlcxx\module.hpp(183,91): warning C4251: 'jlcxx:::FunctionWrapperBase::m_return_type': struct 'std::pair<jl_datatype_t *, jl_datatype_t *>' needs to have dll-interface to be used by clients of class 'jlcxx::FunctionWrapperBase' [D:\cxxwrap-template\build\foo.vcxproj]

It looks like the dlls I just created for libcxxwrap-julia are not found when I build cxxwrap-template.

Only warnings and no errors? These should be harmless, so there should be a new dll in the cxxwrap-template/build directory.

1 Like

Seems there is no error. You are right there is a dll in the folder called D:\cxxwrap-template\build\Debug\foo.dll. There is no bin folder like libcxxwrap-julia. When I try to use the dll in the file testfoo.jl in module ModFoo I have the following error in Julia:

Could not load library "D:\cxxwrap-template\build\Debug\foo.dll"
The specified procedure  could not be found.
Stacktrace:
 [1] dlopen(s::String, flag::UInt32; throw_error::Bool)
...etc

Is this after loading CxxWrap? Most likely this means that the libcxxwrap-julia DLLs can’t be found. This happens if either CxxWrap is not loaded, or loaded but not using an Overrides.toml to direct to the libcxxwrap-julia you built yourself.

1 Like

Here is what I have tried after creating the dlls.

Attempt 1:
-Install CxxWrap by typing add CxxWrap without Overrides.toml file created.
-load Cxxwrap with using CxxWrap which loads ok.
-try to run the module in testfoo.jl file
(I get the error message I sent before)

Attempt 2:
-After removing CxxWrap from attempt 1, I created the Overrides.toml file as described in https://github.com/JuliaInterop/libcxxwrap-julia
-I install again CxxWrap and then I try to load it with using CxxWrap and I get the following error message in Julia:

ERROR: InitError: could not load library "C:\Users\gitboy\.julia\artifacts\...\bin\libcxxwrap_julia_stl.dll"
The specified procedure could not be found.
Stacktrace:
[1] dlopen(s::String, flag::UInt32; throw_error::Bool)
...etc

Some similarity here : InitError: could not load library "...libcxxwrap_julia_stl.dll" on windows

I followed the steps suggested by @hiemstar and got a different error.

Let’s continue the discussion about the Overrides.toml in the issue linked below, can you please post your full Overrides.toml and the paths to the built DLLs there?

https://github.com/JuliaInterop/libcxxwrap-julia/issues/99

1 Like

I finally manage to make it work with both MS compiler and LLVM on windows. I can’t explain what the issue was but basically I needed to have a clean installation of Julia ,i.e. the folder C:\Users\gitboy\.julia must be empty except for the Overrides.toml file in the folder artifacts.
Thank you all for your help.

Use Julia version 1.3.1, otherwise building will fail with “no target architecture” error
@barche I was able to compile the library on windows 10 with MSVC2019 and julia 1.8.3. Do you actually mean we shall use Julia version >= 1.3.1?

It started failing in Julia 1.4, but this was fixed either in Julia 1.7 or Julia 1.8, so 1.8 is definitely OK too now!

2 Likes