I am using the examples from PackageCompiler.jl, specifically the MyLib one, and have issues with it. I have compiled MyLib using build.jl file given with it and have created .so files. Now I am trying to use these .so files in Python using ctypes. I have been able to load the .so file successfully, get the function pointer, assign its types, etc. Just one last hurdle and that is segmentation fault occurring while calling the function.
Even the simplest print_hello() that I created as a test does not work and gives the segmentation fault.
I’ve not done this so I can’t help much, except point to a workaround: It seems you really want to compile a library ; to use from Python, but you don’t have to. You can use PythonCall.jl and it will download Julia for you and your Julia package dependencies. And they will be as fast since Julia precompiles them. Only on first use (probably) noticeably slower.
Compiled binaries apps (and also libraries) will not be small (that way), as you’re trying, just about the same size.
Can you show us the error? It seems to me your error may actually be on the Python side. I would first get you Julia code working (then with/from Python), then if you need an app look into AppBundler.jl, then Julia will be your main language (you can still call Python, I believe, it should also then just download Python for you, if you insist on both languages bundled in one distributable file, I think it may be harder, but possible). I would only compile with any method if you’re sure it’s needed and helpful. Then as you do, or several other options, e.g. StaticCompiler.jl (with many limitations, could be best if you really need that and small binaries, or compile to webassembly, then GC not an issue.
PackageCompiler.jl creates a system image in the form of a .so file. However, this is not meant to be loaded directly. You still need to initialize the Julia runtime.
If your primary objective is to use this from Python, I recommend using the (py)juliacall package to initialize thr Julia runtime, passing the system image in as an option.
I am compiling MyLib using PackageCompiler and the generated so file “libmylib.so” I am loading that directly into Python without loading the “libjulia.so”, so initialisation is not done.
Yess, the error is on the Python end: segmentation fault, now when I use the libjulia.so to init Julia, it shows me another error of sys.so not found.
Please check the other reply. Thanks for your response. I am looking for a solution wherein Julia need not be installed, but the code can be transferred into binaries for usage in other language like Python.
Do you have a specific reason for creating the .so file as opposed to create_app which creates an application (.exe on windows) with a bundle of files to be “deployed” independent of Julia being installed on the target machine? Note that, although the documentation is great, the julia_main method required is indeed empty and the ARGs are the normal “magic object” type that are passed in at the command line to be consumed by your actual Julia code.
We have found that to be a more suitable way to Dev in Julia then pass over to python Devs for integration or library exposure since, as to the above, outside of the original machine used to create the app (which is the same for the .so) file it clearly separates the dependency without much more work.
PackageCompiler.jl essentially copies the Julia runtime into the shared library directory, right? If one were to do that, would this approach be viable?
fwiw, @thecodingbugs I tried this approach (in 2019, things may have changed since then, so take this with a grain of salt) and got it to work with a simple Julia program but compilation of a reasonably sized program took a prohibitively long amount of time.
You might be better off using juliacall: Guide · PythonCall & JuliaCall and having the user install Julia. You can even automate the installation of Julia fairly easily using juliaup and Python subprocess commands.
I copied the sys.so file from “Download Julia page, and downloaded Julia binaries”, and other files as well which were required to run the script. Now, still I am getting segmentation fault but now I have more information:
signal(11.1): Segmentation fault
in expression starting at none:0
unknown function (ip: (nil))
Allocations: 2907 (Pool: 2899; Big: 8); GC: 0
I using this : PackageCompiler.jl/examples/MyLib at master · JuliaLang/PackageCompiler.jl · GitHub as an example to see whether it works or not. In this link there is a folder build and in that folder there is a file which Julia provides build.jl I used this file to build the .so file directly for the MyLib package that is provided in the above link. For creation of system image. the file generated is sysimage.so, while in the error before it showed it said cannot find sys.so file, so I suppose both of them are different and once I put sys.so from Julia binaries, it threw the error for so files that are packaged along with the sys.so, so I copied them as well and all the error went away. The segmentation fault error threw shows the memory access of 0 the address was no random hex number but just zeros. I am looking into JuliaCall now, but I need to know if it is any better than using PackageCompiler.jl in terms of speed and simplicity. Let me know. Thanks in advance!
Thr technologies are complimentary. Juliacall makes it easy to load and initialize Julia from Python. PackageCompiler.jl mainly helps you create a system image.
If you compile your code ahead-of-time (AOT) into the system image, then Julia will not have do this compilation later. Then you can invoke this via juliacall.
Notice that even in the example you linked C has to invoke julia_init and relay command line argiments.