How to use locally built LLVM to build Julia?

Internals & Design
1

I was wondering how could one use a locally built LLVM (with some bug fixes) to build Julia? Is there some document that outlines this.
Thanks!

1 Like
2

Setting in my Make.user:

vchuravy@odin ~/b/julia-llvmgit> cat Make.user
DEPS_GIT=1
USE_BINARYBUILDER_LLVM=0

when I then make -C deps get-llvm

vchuravy@odin ~/b/julia-llvmgit> make -C deps get-llvm
make: Entering directory '/home/vchuravy/builds/julia-llvmgit/deps'
git clone -q --mirror --branch julia-14.0.6-2 https://github.com/JuliaLang/llvm-project.git /home/vchuravy/src/julia/deps/srccache/llvm.git/
touch -c /home/vchuravy/src/julia/deps/srccache/llvm.git
make: Leaving directory '/home/vchuravy/builds/julia-llvmgit/deps'

I now have bare checkout in /home/vchuravy/src/julia/deps/srccache/llvm.git/.

Using make -C deps extract-llvm I can convert that bare checkout into a full one

vchuravy@odin ~/b/julia-llvmgit> make -C deps extract-llvm
make: Entering directory '/home/vchuravy/builds/julia-llvmgit/deps'
# try to update the cache, if that fails, attempt to continue anyways (the ref might already be local)
cd /home/vchuravy/src/julia/deps/srccache/llvm.git && git fetch -q https://github.com/JuliaLang/llvm-project.git julia-14.0.6-2:remotes/origin/julia-14.0.6-2
git clone -q --depth=10 --branch julia-14.0.6-2 /home/vchuravy/src/julia/deps/srccache/llvm.git /home/vchuravy/src/julia/deps/srccache/llvm
warning: --depth is ignored in local clones; use file:// instead.
cd /home/vchuravy/src/julia/deps/srccache/llvm && git remote set-url origin https://github.com/JuliaLang/llvm-project.git
touch -c /home/vchuravy/src/julia/deps/srccache/llvm/CMakeLists.txt # old target
echo 1 > /home/vchuravy/src/julia/deps/srccache/llvm/source-extracted
# try to update the cache, if that fails, attempt to continue anyways (the ref might already be local)
cd /home/vchuravy/src/julia/deps/srccache/llvm.git && git fetch -q https://github.com/JuliaLang/llvm-project.git julia-14.0.6-2:remotes/origin/julia-14.0.6-2
cd /home/vchuravy/src/julia/deps/srccache/llvm && git fetch -q /home/vchuravy/src/julia/deps/srccache/llvm.git remotes/origin/julia-14.0.6-2:remotes/origin/julia-14.0.6-2
cd /home/vchuravy/src/julia/deps/srccache/llvm && git checkout -q --detach julia-14.0.6-2
\033[33;1m==> warning: SHA1 hash did not match llvm.version file\033[0m
echo 1 > /home/vchuravy/src/julia/deps/srccache/llvm/source-extracted

Note the /home/vchuravy/src/julia/deps/srccache/llvm/source-extracted marker file.
/home/vchuravy/src/julia/deps/srccache/llvm/ now contains a proper checkout that you can work on or modify.

Using make -C deps configure-llvm to setup the build.

vchuravy@odin ~/b/julia-llvmgit> make -C deps configure-llvm
...
-- Build files have been written to: /home/vchuravy/builds/julia-llvmgit/deps/llvm/build_Release
echo 1 > llvm/build_Release/build-configured
make: Leaving directory '/home/vchuravy/builds/julia-llvmgit/deps'

Now what happens when I modify a source file.

vchuravy@odin ~/b/julia-llvmgit> touch /home/vchuravy/src/julia/deps/srccache/llvm/llvm/CMakeLists.txt 
vchuravy@odin ~/b/julia-llvmgit> make -C deps configure-llvm
make: Entering directory '/home/vchuravy/builds/julia-llvmgit/deps'
make: Nothing to be done for 'configure-llvm'.
make: Leaving directory '/home/vchuravy/builds/julia-llvmgit/deps'

As we can see the “touch” isn’t picked up. This is due to Julia using “Make markers” so we also need:

touch /home/vchuravy/src/julia/deps/srccache/llvm/source-extracted

Hope that helps!

6 Likes