Documention around Julia build

General Usage
3

Overview of julia’s build process

# v1.8.3
make O=julia-wsl configure
make -C julia-wsl/ -j4  # build all things

We build julia in the folder julia-wsl.

Build the order of targets

You can build these targets one by one, making it easy to observe each step of the action.

  • julia-deps
  • julia-stdlib
  • julia-base
  • julia-cli-release
  • julia-src-release
  • julia-sysimg-release
  • julia-base-cache

make -C julia-wsl/ julia-deps

Equivalent to make -C julia-wsl/deps/.

Most of the deps\Makefile file is making conditional judgments about whether to use the system’s, or julia’s compiled dependencies.
There are a number of includes at the end of the makefile, and each individually imported .mk file is responsible for building a given dependency.

In general, most of the dependencies are pre-compiled in advance by BinaryBuilder.jl. When executing make, most of the dependencies are simply downloaded as pre-compiled artifacts, while the remaining few dependencies need to be compiled.

Direct Download

BinaryBuilder pre-compiled dependencies are downloaded and unpacked directly to julia-wsl\usr\

The compilation script used for precompilation is found in the BB compilation configuration repo by the dependency name: GitHub - JuliaPackaging/Yggdrasil: Collection of builder repositories for BinaryBuilder.jl

e.g.:

Source code compilation

Dependencies that need to be compiled from source code:

  • utf8proc
  • libwhich
  • patchelf

make -C julia-wsl/ julia-stdlib

Equivalent to make -C julia-wsl/stdlib/

Many of the julia standard libraries have been moved out of the main julia repo, so they need to be downloaded.
There are also some standard libraries that have binary dependencies and need to be downloaded.

There are two types of julia packages

  • The normal source-only packages, with .jl as a suffix
  • thin wrappers of binary dependencies, usually automatically generated, with _jll.jl as a suffix.

e.g.: LibCURL.jl, LibCURL_jll.jl

Standard libraries to be downloaded

Print the list of standard libraries to be downloaded:
make -C julia-wsl/stdlib/ print-STDLIBS_EXT

Pkg Statistics LibCURL Downloads ArgTools Tar NetworkOptions SuiteSparse SparseArrays SHA

  • The version number (commit) is specified by the corresponding stdlib\*.version file
  • Download cache path: stdlib\srccache
  • Installation path: julia-wsl\stdlib

JLL binary dependencies to be downloaded

Print all JLL packages:
make -C julia-wsl/stdlib/ print-JLL_NAMES

MozillaCACerts_jll dSFMT_jll GMP_jll LibCURL_jll LibGit2_jll libLLVM_jll LibSSH2_jll LibUV_jll MbedTLS_jll MPFR_jll nghttp2_jll libblastrampoline_jll OpenBLAS_jll OpenLibm_jll p7zip_jll PCRE2_jll SuiteSparse_jll Zlib_jll LLVMLibUnwind_jll CompilerSupportLibraries_jll LibUnwind_jll
  • The corresponding thin wrapper is already located in the stdlib\ folder
  • The version number is specified by the corresponding stdlib\*_jll\Project.toml
  • Download and unpack and install it to julia-wsl\usr\

make -C julia-wsl/ julia-base

Equivalent to make -C julia-wsl/base/

A number of symbolic links are created, generating a number of .jl files containing constants.

make -C julia-wsl/ julia-cli-release

Equivalent to make -C julia-wsl/cli/ release

Build cli and loader.

make -C julia-wsl/ julia-src-release

Equivalent to make -C julia-wsl/src/ release

Build the core of julia.

Folder build order:

  • julia/src
  • julia/src/support
  • julia/src/flisp

make -C julia-wsl/ julia-sysimg-release

Warn: I don’t know much about this step, so I may not be describing it accurately.

Execute sysimage.mk to build the system image.
Compile the Base+stdlib .jl file into a binary.

This step also generates the REPL pre-compilation statements and executes them.

make -C julia-wsl/ julia-base-cache

Write the sys source cache in format readable by Base._read_dependency_src
—— etc\write_base_cache.jl

final

You can run make -C julia-wsl/ -j4. It will build two more test targets.
After that you can execute . /julia to open the julia REPL.

4 Likes