Compiling and building binaries from your julia code

General Usage
21

Nice! Care to make a pull request?

Also, those line-31 from and to lines look the same.

22

A PR would be much appreciated!

-viral

23

At the end of line-31 I added: -Wl,--export-all-symbols

24

Ok, give me some time and I’ll create a PR for this and other minor changes.

However I still cannot get a working executable, I tried also with x86_64-w64-mingw32-gcc version 7.1.0 x86_64-win32-seh-rev2 with no luck.

I simplified the hello.jl code to the following:

module mystuff
Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
    println("hello, world")
    return 0
end
end

It compiles without errors but I do not get “hello, word” printed out.

25

I finally found out why!! I did not have in my path the Julia bin directory. I copied all the dll files from the Julia bin directory to my building directory and now I get “hello, world”… cool!!
What fooled me is that I was not getting any error messages, but yeah, that was easy…

26

Now that it works I have the following questions:

  • Should not the binary emit an error when it does not find the required dll files in path?
  • As the -Wl,--export-all-symbols flag is required on Windows, should it be added to the Windows ldflags in julia-config.jl?
  • As the -Wl,-rpath,\$ORIGIN flag is required on all platforms, should it be added to the ldlibs in julia-config.jl?

Thanks

27

Hi, I am adding some improvements to the juliac.jl script, e.g. to make it work on Windows and to allow building on a different directory. Sorry I still need a few days and then I will be able to start submitting PRs.
However I have a question: why did you allow to specify the Julia installation Path and Julia Package Directory? Would not be better to always run the juliac.jl script using the Julia installation for which you want to compile, and then detect automatically those directories based on JULIA_HOME and Pkg.dir()?
If I have two Julia installations A and B, and run the juliac.jl script using installation A but specifying the Julia installation Path and Julia Package Directory of installation B, the actual vers = "v$(VERSION.major).$(VERSION.minor)" at line 19 of the juliac.jl script will be for installation A. Could not this lead to issues?
Thanks

28

Will this finally allow me to easily build julia executables I can call from within R? I’ve been waiting forever for that feature.

1 Like
29

I am now working to add Julia v0.7 support to static-julia.

On the README I read that:

With Julia 0.7, a single large binary can be created, which does not require the driver program to load the shared library. An example of that is in program2.c, where the image file is the binary itself."

Can you give me some hints on how this is supposed to work?

This command:

gcc -m64 -o hello program2.c hello.o <cflags> <ldflags> <ldlibs> -Wl,-rpath,$ORIGIN

fails with the following error:

/usr/bin/ld: hello.o: undefined reference to symbol ‘trunc@@GLIBC_2.2.5’
//lib/x86_64-linux-gnu/libm.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

With this command I get an executable:

gcc -m64 -o hello program2.c libhello.so <cflags> <ldflags> <ldlibs> -Wl,-rpath,$ORIGIN

which is clearly not a “single large binary”, and when I run it I get the following error:

WARNING:
signal (11): Segmentation fault
while loading no file, in expression starting on line 0
ctx_switch at /home/centos/buildbot/slave/package_tarball64/build/src/task.c:363
Allocations: 93395 (Pool: 93316; Big: 79); GC: 0
Segmentation fault

30

I’m having a bit of trouble having 0.7 to work with this, with packages being in flux. Perhaps the right time to visit 0.7 is after the 0.7 alpha release. IIRC, your first command below was how I had it working at one point.

For now, perhaps the best thing to do is to get it working well on 0.6. Thank you for all the PRs that already make it much easier to work with.

-viral

31

Ok, in the next days I am going to create a last PR with preliminary Julia v0.7 support in a separate script juliac_v07.jl, so that I do not forget what I have in mind, then we will test it once Julia v0.7 alpha is released.

32

See line 40 of program2.c:

jl_options.image_file = "libhello.so";

Should this line be modified as follows or removed altogether for Julia v0.7?

jl_options.image_file = "hello.o";

Thanks

33

One question: Could this tool be useful to create a shared library to include Julia functions in Matlab? Or is the call overhead too high?

34

I need to call a function 10 000 times per second, so loading the Julia library each time might be too much overhead.

35

I really have no idea if the overhead would be too high or not. Haven’t used matlab in many years now! Probably should be easy to hook up the hello.jl example and see what you get.

-viral

3 Likes
36

To load an .so file into Matlab I also need a header file. Where can I find/
what can I use as hello.h file?

37

you could also check out https://github.com/twadleigh/mexjulia

38

Well, this is not in a usable state for Linux users:
https://github.com/twadleigh/mexjulia/issues/32

So I need a different solution.

39

Hi, please could anyone help with the “static-julia” compilation on Windows?
I’ve downloaded the latest version of the “static-julia” project. Trying to compile the “hello” example I’m getting error listed below. Thanks.

Build shared library "libhello.dll":
`'c:\cygwin64\bin\x86_64-w64-mingw32-gcc.exe' -m64 -shared -o libhello.dll hello.o -std=gnu99 '-IC:\Users\Igor Cerovsky\AppData\Local\Julia-0.6.0\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Users\Igor Cerovsky\AppData\Local\Julia-0.6.0\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -Wl,--export-all-symbols`
C:/cygwin64/lib/gcc/x86_64-w64-mingw32/6.4.0/collect2.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory
ERROR: LoadError: failed process: Process(`'c:\cygwin64\bin\x86_64-w64-mingw32-gcc.exe' -m64 -shared -o libhello.dll hello.o -std=gnu99 '-IC:\Users\Igor Cerovsky\AppData\Local\Julia-0.6.0\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Users\Igor Cerovsky\AppData\Local\Julia-0.6.0\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -Wl,--export-all-symbols`, ProcessExited(1)) [1]
1 Like
40

This is pretty fantastic, thanks @viralbshah! So I understand that there’s a pretty big (~34M on my machine for a program that just does a single println) ‘system image’ shared lib produced that I guess (and output of nm libhello.dylib seems to confirm) includes every function that could possibly be called out of the packages used by the program being compiled. Some questions related (and not) to this arise in my mind:

  1. Aside from the overhead of the library having a ton of symbols that won’t be used at runtime, is the generated code guaranteed to be free of ‘startup overhead’ as is seen in the REPL? Put another way, is every execution of the same program identical (modulo OS-level cacheing etc)?

  2. Could the library (or single binary in 0.7) be stripped of the unused symbols?

  3. Does garbage collection act the same in these binaries/libraries as when executing a .jl script? Would the GC present a barrier to applications where deterministic timing is important, e.g. real-time controllers?

  4. Can the generated binary make use of threads? In the parallel-processing sense (map-reduce etc) as well as long running task threads. I haven’t gotten into using threading in Julia yet so apologies if this might be more of a general Julia question!

As you might guess from my questions, I’m excited at the prospect of using Julia to develop something like an embedded real-time control system and static-julia seems like a it might be a good starting point.