JSON3 fails to precompile due to embedded NULs in C strings

Hi all,
When trying to precompile JSON3.jl on Julia 1.11.4, the following error occurs:
Is there a known fix for this problem?
Thank you for your support and guidance.

ERROR: LoadError: ArgumentError: embedded NULs are not allowed in C strings: "{\"a\":{\"a\"\0\"b\"\x15\0"
Stacktrace:
  [1] unsafe_convert
    @ ./strings/cstring.jl:85 [inlined]
  [2] stat(path::String)
    @ Base.Filesystem ./stat.jl:174
  [3] isfile
    @ ./stat.jl:494 [inlined]
  [4] read_json_str(json::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/utils.jl:240
  [5] #read#5
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:33 [inlined]
  [6] read
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:31 [inlined]
  [7] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:68
  [8] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55
  [9] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:84
 [10] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55 [inlined]
 [11] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54 [inlined]
 [12] pretty(out::IOBuffer, x::JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54
 [13] macro expansion
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:19 [inlined]
 [14] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
 [15] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:102
 [16] include(mod::Module, _path::String)
    @ Base ./Base.jl:557
 [17] include(x::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
 [18] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:176
 [19] include
    @ ./Base.jl:557 [inlined]
 [20] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2881
 [21] top-level scope
    @ stdin:6
in expression starting at /path/to/.julia/packages/JSON3/xG3dM/src/workload.jl:3
in expression starting at /path/to/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
in expression starting at stdin:6

Hi!

What OS is this on, and how did you install Julia? This sort of weird error (that the rest of us don’t get) often comes from installing Julia directly from a Linux distribution package instead of using juliaup. If that’s your case, I’d recommend uninstalling this Julia and installing it using juliaup.

If that’s not your issue, it would be useful to know how exactly you’re installing JSON3.jl (what command you’re using), and whether you’ve tried to do that in the past (eg. an old precompile may have been interrupted leaving it this way). Any detail and context around this would be helpful.

Thanks!

I installed Julia using Juliaup and ran ] add JSON3, so I believe I followed the standard procedure.
Interestingly, everything works fine on WSL, but on the supercomputer, I encounter issues.
I’m trying to use HighVoronoi.jl and PencilFFTs.jl, but both fail to precompile due to the issue mentioned above on the supercomputer.
I also tried to install PencilFFTs about a year ago, but it failed to precompile for the same reason.
So I believe this isn’t a Julia version issue, but rather a more general problem.

julia> versioninfo()
Julia Version 1.11.4
Commit 8561cc3d68d (2025-03-10 11:36 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 384 Ă— AMD EPYC 9654 96-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver4)
Threads: 1 default, 0 interactive, 1 GC (on 384 virtual cores)
Environment:
  LD_LIBRARY_PATH = /apps/t4/rhel9/free/cudnn/9.8.0/cuda/12/lib:/apps/t4/rhel9/free/hdf5-parallel/1.14.3/nvhpc24.1/openmpi5.0.2/lib:/apps/t4/rhel9/free/openmpi/5.0.2/nvhpc/lib:/apps/t4/rhel9/free/ucx/1.16.0-gcc/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/comm_libs/nvshmem/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/comm_libs/nccl/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/math_libs/lib64:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/extras/qd/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda/lib64
(MyPackage) pkg> add JSON3
   Resolving package versions...
    Updating `$PRIVATE/Project.toml`
  [0f8b85d8] + JSON3 v1.14.2
    Updating `$PRIVATE/Manifest.toml`
  [0f8b85d8] + JSON3 v1.14.2
  [856f2bd8] + StructTypes v1.11.0
Precompiling project...
  âś— JSON3
  0 dependencies successfully precompiled in 26 seconds. 398 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:

JSON3

Failed to precompile JSON3 [0f8b85d8-7281-11e9-16c2-39a750bddbf1] to "$HOME/.julia/compiled/v1.11/JSON3/jl_Vua5se".
ERROR: LoadError: ArgumentError: embedded NULs are not allowed in C strings: "{\"a\":{\"a\"\0\"b\"\x14\0"
Stacktrace:
  [1] unsafe_convert
    @ ./strings/cstring.jl:85 [inlined]
  [2] stat(path::String)
    @ Base.Filesystem ./stat.jl:174
  [3] isfile
    @ ./stat.jl:494 [inlined]
  [4] read_json_str(json::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/utils.jl:240
  [5] #read#5
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:33 [inlined]
  [6] read
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:31 [inlined]
  [7] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:68
  [8] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55
  [9] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:84
 [10] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55 [inlined]
 [11] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54 [inlined]
 [12] pretty(out::IOBuffer, x::JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54
 [13] macro expansion
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:19 [inlined]
 [14] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
 [15] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:102
 [16] include(mod::Module, _path::String)
    @ Base ./Base.jl:557
 [17] include(x::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
 [18] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:176
 [19] include
    @ ./Base.jl:557 [inlined]
 [20] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2881
 [21] top-level scope
    @ stdin:6
in expression starting at $HOME/.julia/packages/JSON3/xG3dM/src/workload.jl:3
in expression starting at $HOME/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
in expression starting at stdin:```

Ah, it’s an architecture dependent bug.

Searching for this error on github didn’t lead to any especially relevant discussion. ArgumentError: embedded NULs are not allowed in C strings · Issue #594 · julia-vscode/LanguageServer.jl · GitHub seems the closest, but that too is unresolved and is under julia-vscode.

The error’s source (in this case) seems to be near the end of this line:

which is somehow getting misread as that string with NULs "{\"a\":{\"a\"\0\"b\"\x14\0".

I’ve added the hpc tag to the post since you mentioned this happens on the supercomputer. Hopefully someone with more experience on that side of things chimes in. If not, this is probably worth creating an issue at julialang/julia for - since this isn’t a single package issue, but happens across a few different ones.

There is a bug in the pretty function in JSON3. Could you please open a issue with this error? GitHub - quinnj/JSON3.jl

XRef: Internal use of `JSON3.read` function may interact with the file system · Issue #309 · quinnj/JSON3.jl · GitHub

I think there is something else going on here as well, but does Avoid file name vs contents confusion in `pretty` by nhz2 · Pull Request #310 · quinnj/JSON3.jl · GitHub fix the issue you are having?

You can try it with:

pkg> add https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read

Thanks, everyone!
I tried pkg> add https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read,
but I ran into a different error.
I plan to open an issue about this later, but could you take a look at the error first?

(MyPackage) pkg> add https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read
     Cloning git-repo `https://github.com/nhz2/JSON3.jl.git`
    Updating git-repo `https://github.com/nhz2/JSON3.jl.git`
   Resolving package versions...
    Updating `$PRIVATE/Project.toml`
  [0f8b85d8] ~ JSON3 v1.14.2 ⇒ v1.14.2 `https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read`
    Updating `$PRIVATE/Manifest.toml`
  [0f8b85d8] ~ JSON3 v1.14.2 ⇒ v1.14.2 `https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read`
Precompiling project...
  âś— JSON3
  0 dependencies successfully precompiled in 24 seconds. 398 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:

JSON3

Failed to precompile JSON3 [0f8b85d8-7281-11e9-16c2-39a750bddbf1] to "$HOME/.julia/compiled/v1.11/JSON3/jl_SciF2v".
ERROR: LoadError: ArgumentError: invalid JSON at byte position 10 while parsing type JSON3.Object: ExpectedSemiColon
{"a":{"a""b"

Stacktrace:
  [1] invalid(error::JSON3.Error, buf::Base.CodeUnits{UInt8, String}, pos::Int64, T::Type)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/JSON3.jl:30
  [2] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{JSON3.Object}, checkint::Bool; kw::@Kwargs{allow_inf::Bool})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:318
  [3] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{Any}, checkint::Bool; allow_inf::Bool)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:98
  [4] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{JSON3.Object}, checkint::Bool; kw::@Kwargs{allow_inf::Bool})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:287
  [5] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{Any}, checkint::Bool; allow_inf::Bool)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:98
  [6] read!
    @ ~/.julia/packages/JSON3/9OLdM/src/read.jl:96 [inlined]
  [7] parse(bytes::Base.CodeUnits{UInt8, String}; jsonlines::Bool, numbertype::Nothing, kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:61
  [8] parse
    @ ~/.julia/packages/JSON3/9OLdM/src/read.jl:36 [inlined]
  [9] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:68
 [10] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:55
 [11] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:84
 [12] pretty
    @ ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:55 [inlined]
 [13] pretty
    @ ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:54 [inlined]
 [14] pretty(out::IOBuffer, x::JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:54
 [15] macro expansion
    @ ~/.julia/packages/JSON3/9OLdM/src/workload.jl:19 [inlined]
 [16] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
 [17] top-level scope
    @ ~/.julia/packages/JSON3/9OLdM/src/workload.jl:102
 [18] include(mod::Module, _path::String)
    @ Base ./Base.jl:557
 [19] include(x::String)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/JSON3.jl:1
 [20] top-level scope
    @ ~/.julia/packages/JSON3/9OLdM/src/JSON3.jl:176
 [21] include
    @ ./Base.jl:557 [inlined]
 [22] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2881
 [23] top-level scope
    @ stdin:6
in expression starting at $HOME/.julia/packages/JSON3/9OLdM/src/workload.jl:3
in expression starting at $HOME/.julia/packages/JSON3/9OLdM/src/JSON3.jl:1
in expression starting at stdin:

Thank you for trying that. Do you get the same error if you use Julia 1.10?

Thanks!!

After some investigation, I found:

  • The error does not occur when using julia +lts on the supercomputer.
  • The error also disappears when I run again with julia +1.11.4.
  • However, if I launch Julia with the --check-bounds=no option, the error consistently occurs — both on my local machine and on the supercomputer.

So I strongly suspect that --check-bounds=no is the cause of the issue.
It turns out the supercomputer was not the cause of the issue — sorry for the confusion.

1 Like

Well, that explains it. I can reproduce it locally with julia --check-bounds=no.

This @boundscheck is the “reason”: JSON3.jl/src/write.jl at fb220e513d8c43e6a698e2c863da2a5e9bcab2e3 · quinnj/JSON3.jl · GitHub When you run julia with --check-bounds=no, this line is elided and the buffer is not reallocated (by the @check macro).

Ref: Remove inappropriate `@boundscheck` by barucden · Pull Request #311 · quinnj/JSON3.jl · GitHub

2 Likes

This is a great example of why combining --check-bounds=no with third-party packages is a really bad idea. Packages rely on bounds checking for correctness because it is a standard feature of Julia. I personally have never even tested if the packages I work on work with --check-bounds=no.

2 Likes