3,000+ files and 1.4GB build directory with PackageCompiler. What am I doing wrong?

[basically copy/pasted from my post on reddit which can be found here]

A few screenshots to illustrate what I’m dealing with: https://imgur.com/a/2YRYkis

As the title says, I’m a bit lost here.

I took the example hello.jl, removed the lines to do with ascii plotting, and went to compile it. The resultant hello program works fine, prints “hello, world” as expected, then ends.

Why is the builddir so huge, with so many unnecessary files (Qt files bundled?).

Currently running it inside a VM, on Manjaro, with Juno as my editor. Happy to provide any other details as need be.

1 Like
  • Does your hello.jl contain any using or import commands?
  • Is there a Project.toml or Manifest.toml file in same directory as hello.jl?
  • Have you cleaned out the builddir then tried the build? Could the files could be from old bulid attempts?

My guess would be that something is pulling in those extra modules. When I try to compile my hello.jl i get two files hello.a and hello.so . But I also get a build error:

/usr/bin/ld: /tmp/ccbTh2qc.o: in function `main':
program.c:(.text+0x153): undefined reference to `julia_main'
collect2: error: ld returned 1 exit status

So I’m guessing I’m missing some sort of setup…

  1. No, I took the using UnicodePlots line out, along with everything relating to it.

  2. There were no secondary build files used or present. Literally just the modified hello.jl file inside a directory.

  3. See above, there was no pre-existing builddir directory present.

I thought the same as you, that juliac.jl was for some reason pulling in every possible dependency it could think of. But after looking through the docs I couldn’t find anything that would cause that - nor anything to stop it from happening either.

As for your error though, are you using a julia_main() function as the entry point? Apparently it’s required here. See here

Okay thank you for that, it’s building now.

What I think is happening is happening is everything from /usr/lib and /lib is copied into builddir. This is probably unnecessary, but makes it easier to link with the libraries needed, would be my guess. I did an sha256sum on a few of the files and they match the system files.

The actual file generated appears to be an 18k file with a 130mb library. The list of libraries being linked in isn’t bad (so it didn’t link with EVERYTHING). :slight_smile:

[pixel@devil ~]$ ldd builddir/test
	linux-vdso.so.1 (0x00007ffc2f922000)
	test.so => /home/pixel/builddir/test.so (0x00007f8ce647c000)
	libjulia.so.1 => /usr/lib/libjulia.so.1 (0x00007f8ce5f74000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f8ce5db1000)
	libutf8proc.so.2 => /usr/lib/libutf8proc.so.2 (0x00007f8ce5d65000)
	libLLVM-6.0.so => /usr/lib/julia/libLLVM-6.0.so (0x00007f8ce3217000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f8ce3212000)
	librt.so.1 => /usr/lib/librt.so.1 (0x00007f8ce3205000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f8ce31e4000)
	libunwind.so.8 => /usr/lib/libunwind.so.8 (0x00007f8ce31ca000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f8ce2fe2000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f8ce2e9c000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f8ce2e82000)
	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8ced9cb000)
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007f8ce2c5a000)

Good to know that someone else is having the same experience that I am at least.

Now we just need to figure out how to tell the compiler not to bundle all the unnecessary parts, and to keep the 130MB library file down a bit.

Yeah and I thought go programs compile to huge executables. But the Julia executable build is still in it’s infancy, so it should improve over time. I’m also curious about the link to libjulia and if that will be “builtin” at some point, of if you will need to distribute julia if you distribute an executable.