Current state of Cxx.jl?

Thank you! I missed that thread.

Hi, are you still there?

I’m going to merge this PR, but before that, I need more volunteers to test it. It would be great if you could give some feedback. :slight_smile:

4 Likes

Definitely, but it’ll take some time as I’m not really experienced with c++ or Cxx itself. (But I happily help!)

1 Like

Installing the a package (pkg> add https://github.com/Gnimuc/Cxx.jl#BB2) went fine, and also using Cxx.
I tried the first and second examples from the Cxx README, and they were also fine. I got an error at the third one.
Then I tried: pkg> test Cxx which also errored. The whole session is below.

C:\Users\cstamas\Documents\GIT>mkdir cxxtest

C:\Users\cstamas\Documents\GIT>cd cxxtest

C:\Users\cstamas\Documents\GIT\cxxtest>julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.1) pkg> activate .

(cxxtest) pkg> add https://github.com/Gnimuc/Cxx.jl#BB2
  Updating registry at `C:\Users\cstamas\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `https://github.com/Gnimuc/Cxx.jl`
  Updating git-repo `https://github.com/Gnimuc/Cxx.jl`
 Resolving package versions...
  Updating `C:\Users\cstamas\Documents\GIT\cxxtest\Project.toml`
  [a0b5b9ef] + Cxx v0.2.0+ #BB2 (https://github.com/Gnimuc/Cxx.jl)
  Updating `C:\Users\cstamas\Documents\GIT\cxxtest\Manifest.toml`
  [b99e7846] + BinaryProvider v0.5.3
  [a0b5b9ef] + Cxx v0.2.0+ #BB2 (https://github.com/Gnimuc/Cxx.jl)
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  Building Cxx → `C:\Users\cstamas\.julia\packages\Cxx\cSkk4\deps\build.log`

julia> using Cxx
[ Info: Precompiling Cxx [a0b5b9ef-44b7-5148-a2d1-f6db19f3c3d2]

julia> cxx""" #include<iostream> """
true

julia> cxx"""
       void mycppfunction() {
       int z = 0;
       int y = 5;
       int x = 10;
       z = x*y + 2;
       std::cout << "The number is" << z << std::endl;
       }
       """
true

julia> julia_function() = @cxx mycppfunction()
julia_function (generic function with 1 method)

julia> julia_function()
The number is52

julia> jnum = 10
10

julia> cx
ERROR: UndefVarError: cx not defined

julia> cxx"""
                  void printme(int x) {
                                std::cout << x << std::endl;
                                           }
                                                  """
true

julia> @cxx printme(jnum)
10

julia> @cxx printme(100)
100

julia> cxx"""
                void printme(const char *name) {
                            // const char* => std::string
                                        std::string sname = name;
                                                    // print it out
                                                                std::cout << sname << std::endl;
                                                                         }
                                                                              """
true

julia> @cxx printme(pointer("John"))
LLVM ERROR: Program used external function '__gxx_personality_v0' which could not be resolved!

C:\Users\cstamas\Documents\GIT\cxxtest>julia --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(cxxtest) pkg> test Cxx
   Testing Cxx
 Resolving package versions...
    Status `C:\Users\cstamas\AppData\Local\Temp\jl_2E46.tmp\Manifest.toml`
  [b99e7846] BinaryProvider v0.5.3
  [a0b5b9ef] Cxx v0.2.0+ #BB2 (https://github.com/Gnimuc/Cxx.jl)
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x0 -- unknown function (ip: 0000000000000000)
in expression starting at C:\Users\cstamas\.julia\packages\Cxx\cSkk4\test\runtests.jl:7
unknown function (ip: FFFFFFFFFFFFFFFF)
CloneFunctionAndAdjust at C:\Users\cstamas\.julia\packages\Cxx\cSkk4\deps\usr\bin\libcxxffi.DLL (unknown line)
ReplaceFunctionForDecl at C:\Users\cstamas\.julia\packages\Cxx\cSkk4\deps\usr\bin\libcxxffi.DLL (unknown line)
#ReplaceFunctionForDecl#5 at C:\Users\cstamas\.julia\packages\Cxx\cSkk4\src\clangwrapper.jl:385 [inlined]
#ReplaceFunctionForDecl at .\none:0
unknown function (ip: 000000001F2C2CC8)
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1864
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
do_call at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:323
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:411
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:362 [inlined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:755
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:689
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:689
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:885
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 000000000608EA8F)
unknown function (ip: 00000000000000A3)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:764
jl_parse_eval_all at /home/Administrator/buildbot/worker/package_win64/build/src\ast.c:883
jl_load at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:826 [inlined]
jl_load_ at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:833
include at .\boot.jl:326 [inlined]
include_relative at .\loading.jl:1038
include at .\sysimg.jl:29
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
include at .\client.jl:403
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1864
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
do_call at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:323
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:411
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:362 [inlined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:755
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:885
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 0000000005B106EF)
unknown function (ip: FFFFFFFFFFFFFFFF)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:764
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:713
jl_toplevel_eval at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:773 [inlined]
jl_toplevel_eval_in at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:793
eval at .\boot.jl:328
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
exec_options at .\client.jl:243
_start at .\client.jl:436
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2219
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1571 [inlined]
true_main at /home/Administrator/buildbot/worker/package_win64/build/ui\repl.c:96
wmain at /home/Administrator/buildbot/worker/package_win64/build/ui\repl.c:217
__tmainCRTStartup at /usr/src/debug/mingw64-x86_64-runtime-5.0.3-1/crt\crtexe.c:329
mainCRTStartup at /usr/src/debug/mingw64-x86_64-runtime-5.0.3-1/crt\crtexe.c:212
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 16304048 (Pool: 16301277; Big: 2771); GC: 38
ERROR: Package Cxx errored during testing

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, ivybridge)

(cxxtest) pkg> st
    Status `C:\Users\cstamas\Documents\GIT\cxxtest\Project.toml`
  [a0b5b9ef] Cxx v0.2.0+ #BB2 (https://github.com/Gnimuc/Cxx.jl)

I’m happy to try anything else, just don’t know what.

2 Likes

Those errors are expected since Windows support is still experimental for now. Thanks for the feedback! :wink:

1 Like

I tested at work on a Linux server (Ubuntu 16.04.6). The installation went fine and also all the examples I tried (1 to 7). (Although at the fourth example the function returned immediately and didn’t wait for a key.) Testing the package was also succesful.

(cxxtest) pkg> test Cxx
   Testing Cxx
 Resolving package versions...
    Status `/tmp/tmpmOcWmI/Manifest.toml`
  [b99e7846] BinaryProvider v0.5.3
  [a0b5b9ef] Cxx v0.2.0+ #BB2 (https://github.com/Gnimuc/Cxx.jl)
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]
Test Summary:                   | Pass  Total
import without using Cxx (#316) |    2      2
foo
foo
foo
foo
foo
foo
foo
foo
foo
foo
1
2
3
4
5
6
7
8
9
10
Test Summary: | Pass  Total
StdString     |    3      3
Test Summary: | Pass  Total
StdVector     |   70     70
e = (class std::length_error &) {
}

Test Summary: | Pass  Total
Exceptions    |    2      2
   Testing Cxx tests passed

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, broadwell)

Should I try any other tests (or examples)?

I’m really happy to see that this is now in the JuliaInterop github group, when did that happen? Hopefully that is an indication that it will be more realistic to actually merge PR’s in the near future.

2 Likes

Thanks for this work!

I can confirm that everything went well when I tested on my system (Debian testing) : installation, examples from the README, and test suite.

1 Like

It’s working great for me in both Ubuntu 18.04 and MacOS!

2 Likes

Is it necessary to install all of Xcode to run Cxx? I have the command-line tools, but ]test Cxx fails with ERROR: LoadError: LoadError: Could not find C++ standard library. Is XCode installed?. I’d rather not install all of Xcode if I can avoid it.

Thanks.

1 Like

I tried building Cxx, but it ate up 96 GB of ram before running out and throwing an error,
That computer has 32 GB of RAM, and I thought 64 GB of swap would be enough.

do you know another way to find those libc++ headers?

did you build Cxx from source or use pre-built binaries? Could you post those errors?

Trying to build Cxx master on a 5 day old Julia master, with a different computer that has 8 x 16G sticks of RAM (although htop shows 126G total):

┌ Error: Error building `Cxx`, showing the last 100 of log: 
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ virtual memory exhausted: Cannot allocate memory
│ {standard input}: {standard input}: {standard input}: Assembler messages:
│ Assembler messages:
│ Assembler messages:
│ {standard input}:445351: {standard input}:402343: {standard input}:835515: Fatal error: Fatal error: Fatal error: can't close CMakeFiles/clangStaticAnalyzerCheckers.dir/ReturnPointerRangeChecker.cpp.o: memory exhaustedcan't close CMakeFiles/clangStaticAnalyzerCheckers.dir/ObjCSelfInitChecker.cpp.o: memory exhaustedcan't close CMakeFiles/clangStaticAnalyzerCheckers.dir/ObjCSuperDeallocChecker.cpp.o: memory exhausted

I was watching htop to see the memory getting consumed by the build.

You can’t add the release version of Cxx with Julia master because of unsatisfiable requirements:

(v1.3) pkg> add Cxx
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Cxx [a0b5b9ef]:
 Cxx [a0b5b9ef] log:
 ├─possible versions are: [0.0.1-0.0.2, 0.1.0-0.1.1, 0.2.0, 0.3.0] or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions [0.0.1-0.0.2, 0.1.0-0.1.1, 0.2.0, 0.3.0]
 └─restricted by julia compatibility requirements to versions: uninstalled — no versions left

I should just checkout v1.2.

They are installed with the command-line tools:

dpo:~ $ xcode-select --print-path
/Library/Developer/CommandLineTools
dpo:~ $ ls /Library/Developer/CommandLineTools/usr/include/c++/v1
__bit_reference          complex.h                new
__bsd_locale_defaults.h  condition_variable       numeric
__bsd_locale_fallbacks.h csetjmp                  optional
__config                 csignal                  ostream
...
compare                  module.modulemap
complex                  mutex

There’s no such folder as /Library/Developer/CommandLineTools/usr/lib/c++/v1 but libc++.dylib is in /usr/lib.

I’m just wondering if we can get by with the command-line tools only.

1 Like

AFAIK, current Cxx master doesn’t work on Julia master because of this PR. But I think you could manually apply this patch and built Julia without BB2-build enabled.

1 Like

I guess it’s possible since we only need those headers, I’ll submit a PR later on.

I tried checking out that PR and merging in the latest changes, but the same problem. The last few lines:

virtual memory exhausted: Cannot allocate memory
│ make[3]: *** [lib/Parse/CMakeFiles/clangParse.dir/build.make:232: lib/Parse/CMakeFiles/clangParse.dir/ParseTentative.cpp.o] Error 1
│ make[3]: Leaving directory '/home/chriselrod/.julia/dev/Cxx/deps/build/clang-6.0.1'
│ make[2]: *** [CMakeFiles/Makefile2:2481: lib/Parse/CMakeFiles/clangParse.dir/all] Error 2
│ 
│ cc1plus: out of memory allocating 22586734432 bytes after a total of 423505920 bytes
│ make[3]: *** [lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/build.make:648: lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/MallocSizeofChecker.cpp.o] Error 1
│ make[3]: Leaving directory '/home/chriselrod/.julia/dev/Cxx/deps/build/clang-6.0.1'
│ make[2]: *** [CMakeFiles/Makefile2:3732: lib/StaticAnalyzer/Checkers/CMakeFiles/clangStaticAnalyzerCheckers.dir/all] Error 2
│ 
│ cc1plus: out of memory allocating 7083286720 bytes after a total of 2427383808 bytes
│ make[3]: *** [lib/Format/CMakeFiles/clangFormat.dir/build.make:128: lib/Format/CMakeFiles/clangFormat.dir/FormatTokenLexer.cpp.o] Error 1
│ make[3]: Leaving directory '/home/chriselrod/.julia/dev/Cxx/deps/build/clang-6.0.1'
│ make[2]: *** [CMakeFiles/Makefile2:3842: lib/Format/CMakeFiles/clangFormat.dir/all] Error 2
│ make[2]: Leaving directory '/home/chriselrod/.julia/dev/Cxx/deps/build/clang-6.0.1'
│ make[1]: *** [Makefile:130: all] Error 2
│ make[1]: Leaving directory '/home/chriselrod/.julia/dev/Cxx/deps/build/clang-6.0.1'
│ make: *** [BuildBootstrap.Makefile:114: build/clang-6.0.1/lib/libclangCodeGen.a] Error 2
│ [ Info: Building julia source build
│ ERROR: Error while loading expression starting at /home/chriselrod/.julia/dev/Cxx/deps/build.jl:40
│ caused by [exception 1]
│ failed process: Process(`make -j28 -f BuildBootstrap.Makefile BASE_JULIA_BIN=/home/chriselrod/Documents/languages/julia/usr/bin BASE_JULIA_SRC=/home/chriselrod/Documents/languages/julia/usr/bin/../..`, ProcessExited(2)) [2]

Subjectively, I believe it took much longer.
There was one process that lasted a very long time (>7 minutes of 100%CPU when I quit watching) that used over > 80G. Another concurrent process slowly crept up in memory usage, reaching 40G the last time I looked at it.

I have no idea about this. Could you give it a try with only a single thread enabled? make -j1

hi @dpo, could you give this PR a try?

1 Like