JSON3 fails to precompile due to embedded NULs in C strings

0samuraiE · April 28, 2025, 8:43am

Hi all,
When trying to precompile JSON3.jl on Julia 1.11.4, the following error occurs:
Is there a known fix for this problem?
Thank you for your support and guidance.

ERROR: LoadError: ArgumentError: embedded NULs are not allowed in C strings: "{\"a\":{\"a\"\0\"b\"\x15\0"
Stacktrace:
  [1] unsafe_convert
    @ ./strings/cstring.jl:85 [inlined]
  [2] stat(path::String)
    @ Base.Filesystem ./stat.jl:174
  [3] isfile
    @ ./stat.jl:494 [inlined]
  [4] read_json_str(json::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/utils.jl:240
  [5] #read#5
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:33 [inlined]
  [6] read
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:31 [inlined]
  [7] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:68
  [8] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55
  [9] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:84
 [10] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55 [inlined]
 [11] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54 [inlined]
 [12] pretty(out::IOBuffer, x::JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54
 [13] macro expansion
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:19 [inlined]
 [14] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
 [15] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:102
 [16] include(mod::Module, _path::String)
    @ Base ./Base.jl:557
 [17] include(x::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
 [18] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:176
 [19] include
    @ ./Base.jl:557 [inlined]
 [20] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2881
 [21] top-level scope
    @ stdin:6
in expression starting at /path/to/.julia/packages/JSON3/xG3dM/src/workload.jl:3
in expression starting at /path/to/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
in expression starting at stdin:6

digital_carver · April 29, 2025, 8:55am

Hi!

What OS is this on, and how did you install Julia? This sort of weird error (that the rest of us don’t get) often comes from installing Julia directly from a Linux distribution package instead of using juliaup. If that’s your case, I’d recommend uninstalling this Julia and installing it using juliaup.

If that’s not your issue, it would be useful to know how exactly you’re installing JSON3.jl (what command you’re using), and whether you’ve tried to do that in the past (eg. an old precompile may have been interrupted leaving it this way). Any detail and context around this would be helpful.

0samuraiE · April 29, 2025, 11:54am

Thanks!

I installed Julia using Juliaup and ran ] add JSON3, so I believe I followed the standard procedure.
Interestingly, everything works fine on WSL, but on the supercomputer, I encounter issues.
I’m trying to use HighVoronoi.jl and PencilFFTs.jl, but both fail to precompile due to the issue mentioned above on the supercomputer.
I also tried to install PencilFFTs about a year ago, but it failed to precompile for the same reason.
So I believe this isn’t a Julia version issue, but rather a more general problem.

julia> versioninfo()
Julia Version 1.11.4
Commit 8561cc3d68d (2025-03-10 11:36 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 384 × AMD EPYC 9654 96-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver4)
Threads: 1 default, 0 interactive, 1 GC (on 384 virtual cores)
Environment:
  LD_LIBRARY_PATH = /apps/t4/rhel9/free/cudnn/9.8.0/cuda/12/lib:/apps/t4/rhel9/free/hdf5-parallel/1.14.3/nvhpc24.1/openmpi5.0.2/lib:/apps/t4/rhel9/free/openmpi/5.0.2/nvhpc/lib:/apps/t4/rhel9/free/ucx/1.16.0-gcc/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/comm_libs/nvshmem/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/comm_libs/nccl/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/math_libs/lib64:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/extras/qd/lib:/apps/t4/rhel9/isv/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda/lib64

(MyPackage) pkg> add JSON3
   Resolving package versions...
    Updating `$PRIVATE/Project.toml`
  [0f8b85d8] + JSON3 v1.14.2
    Updating `$PRIVATE/Manifest.toml`
  [0f8b85d8] + JSON3 v1.14.2
  [856f2bd8] + StructTypes v1.11.0
Precompiling project...
  ✗ JSON3
  0 dependencies successfully precompiled in 26 seconds. 398 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:

JSON3

Failed to precompile JSON3 [0f8b85d8-7281-11e9-16c2-39a750bddbf1] to "$HOME/.julia/compiled/v1.11/JSON3/jl_Vua5se".
ERROR: LoadError: ArgumentError: embedded NULs are not allowed in C strings: "{\"a\":{\"a\"\0\"b\"\x14\0"
Stacktrace:
  [1] unsafe_convert
    @ ./strings/cstring.jl:85 [inlined]
  [2] stat(path::String)
    @ Base.Filesystem ./stat.jl:174
  [3] isfile
    @ ./stat.jl:494 [inlined]
  [4] read_json_str(json::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/utils.jl:240
  [5] #read#5
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:33 [inlined]
  [6] read
    @ ~/.julia/packages/JSON3/xG3dM/src/read.jl:31 [inlined]
  [7] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:68
  [8] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55
  [9] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:84
 [10] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:55 [inlined]
 [11] pretty
    @ ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54 [inlined]
 [12] pretty(out::IOBuffer, x::JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}})
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/pretty.jl:54
 [13] macro expansion
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:19 [inlined]
 [14] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
 [15] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/workload.jl:102
 [16] include(mod::Module, _path::String)
    @ Base ./Base.jl:557
 [17] include(x::String)
    @ JSON3 ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
 [18] top-level scope
    @ ~/.julia/packages/JSON3/xG3dM/src/JSON3.jl:176
 [19] include
    @ ./Base.jl:557 [inlined]
 [20] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2881
 [21] top-level scope
    @ stdin:6
in expression starting at $HOME/.julia/packages/JSON3/xG3dM/src/workload.jl:3
in expression starting at $HOME/.julia/packages/JSON3/xG3dM/src/JSON3.jl:1
in expression starting at stdin:```

digital_carver · April 29, 2025, 1:14pm

Ah, it’s an architecture dependent bug.

Searching for this error on github didn’t lead to any especially relevant discussion. ArgumentError: embedded NULs are not allowed in C strings · Issue #594 · julia-vscode/LanguageServer.jl · GitHub seems the closest, but that too is unresolved and is under julia-vscode.

The error’s source (in this case) seems to be near the end of this line:

github.com/quinnj/JSON3.jl

src/workload.jl

main


      
          using PrecompileTools
          
          @compile_workload begin
              str = """{"a": 1, "b": "hello, world", "c": [1, 2], "d": true, "e": null, "f": 1.92, "g": {"a": {"a" : "b"}}}"""
          
              JSON3.read(IOBuffer(str))
              json = JSON3.read(str)
              for i in "abcdef"
                  json[i]
              end
              json[:g][:a][:a]
          
              JSON3.read(
                  str,

which is somehow getting misread as that string with NULs "{\"a\":{\"a\"\0\"b\"\x14\0".

I’ve added the hpc tag to the post since you mentioned this happens on the supercomputer. Hopefully someone with more experience on that side of things chimes in. If not, this is probably worth creating an issue at julialang/julia for - since this isn’t a single package issue, but happens across a few different ones.

nhz2 · April 29, 2025, 4:01pm

There is a bug in the pretty function in JSON3. Could you please open a issue with this error? GitHub - quinnj/JSON3.jl

nhz2 · April 29, 2025, 7:09pm

XRef: Internal use of `JSON3.read` function may interact with the file system · Issue #309 · quinnj/JSON3.jl · GitHub

I think there is something else going on here as well, but does Avoid file name vs contents confusion in `pretty` by nhz2 · Pull Request #310 · quinnj/JSON3.jl · GitHub fix the issue you are having?

You can try it with:

pkg> add https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read

0samuraiE · April 29, 2025, 9:35pm

Thanks, everyone!
I tried pkg> add https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read,
but I ran into a different error.
I plan to open an issue about this later, but could you take a look at the error first?

(MyPackage) pkg> add https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read
     Cloning git-repo `https://github.com/nhz2/JSON3.jl.git`
    Updating git-repo `https://github.com/nhz2/JSON3.jl.git`
   Resolving package versions...
    Updating `$PRIVATE/Project.toml`
  [0f8b85d8] ~ JSON3 v1.14.2 ⇒ v1.14.2 `https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read`
    Updating `$PRIVATE/Manifest.toml`
  [0f8b85d8] ~ JSON3 v1.14.2 ⇒ v1.14.2 `https://github.com/nhz2/JSON3.jl.git#nz/fix-use-of-JSON3.read`
Precompiling project...
  ✗ JSON3
  0 dependencies successfully precompiled in 24 seconds. 398 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:

JSON3

Failed to precompile JSON3 [0f8b85d8-7281-11e9-16c2-39a750bddbf1] to "$HOME/.julia/compiled/v1.11/JSON3/jl_SciF2v".
ERROR: LoadError: ArgumentError: invalid JSON at byte position 10 while parsing type JSON3.Object: ExpectedSemiColon
{"a":{"a""b"

Stacktrace:
  [1] invalid(error::JSON3.Error, buf::Base.CodeUnits{UInt8, String}, pos::Int64, T::Type)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/JSON3.jl:30
  [2] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{JSON3.Object}, checkint::Bool; kw::@Kwargs{allow_inf::Bool})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:318
  [3] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{Any}, checkint::Bool; allow_inf::Bool)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:98
  [4] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{JSON3.Object}, checkint::Bool; kw::@Kwargs{allow_inf::Bool})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:287
  [5] read!(buf::Base.CodeUnits{UInt8, String}, pos::Int64, len::Int64, b::UInt8, tape::Vector{UInt64}, tapeidx::Int64, ::Type{Any}, checkint::Bool; allow_inf::Bool)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:98
  [6] read!
    @ ~/.julia/packages/JSON3/9OLdM/src/read.jl:96 [inlined]
  [7] parse(bytes::Base.CodeUnits{UInt8, String}; jsonlines::Bool, numbertype::Nothing, kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/read.jl:61
  [8] parse
    @ ~/.julia/packages/JSON3/9OLdM/src/read.jl:36 [inlined]
  [9] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:68
 [10] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:55
 [11] pretty(out::IOBuffer, str::String, ac::JSON3.AlignmentContext; kw::@Kwargs{})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:84
 [12] pretty
    @ ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:55 [inlined]
 [13] pretty
    @ ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:54 [inlined]
 [14] pretty(out::IOBuffer, x::JSON3.Object{Base.CodeUnits{UInt8, String}, Vector{UInt64}})
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/pretty.jl:54
 [15] macro expansion
    @ ~/.julia/packages/JSON3/9OLdM/src/workload.jl:19 [inlined]
 [16] macro expansion
    @ ~/.julia/packages/PrecompileTools/L8A3n/src/workloads.jl:78 [inlined]
 [17] top-level scope
    @ ~/.julia/packages/JSON3/9OLdM/src/workload.jl:102
 [18] include(mod::Module, _path::String)
    @ Base ./Base.jl:557
 [19] include(x::String)
    @ JSON3 ~/.julia/packages/JSON3/9OLdM/src/JSON3.jl:1
 [20] top-level scope
    @ ~/.julia/packages/JSON3/9OLdM/src/JSON3.jl:176
 [21] include
    @ ./Base.jl:557 [inlined]
 [22] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2881
 [23] top-level scope
    @ stdin:6
in expression starting at $HOME/.julia/packages/JSON3/9OLdM/src/workload.jl:3
in expression starting at $HOME/.julia/packages/JSON3/9OLdM/src/JSON3.jl:1
in expression starting at stdin:

nhz2 · April 29, 2025, 10:04pm

Thank you for trying that. Do you get the same error if you use Julia 1.10?

0samuraiE · April 30, 2025, 1:08pm

Thanks!!

After some investigation, I found:

The error does not occur when using julia +lts on the supercomputer.
The error also disappears when I run again with julia +1.11.4.
However, if I launch Julia with the --check-bounds=no option, the error consistently occurs — both on my local machine and on the supercomputer.

So I strongly suspect that --check-bounds=no is the cause of the issue.
It turns out the supercomputer was not the cause of the issue — sorry for the confusion.

barucden · April 30, 2025, 1:32pm

Well, that explains it. I can reproduce it locally with julia --check-bounds=no.

This @boundscheck is the “reason”: JSON3.jl/src/write.jl at fb220e513d8c43e6a698e2c863da2a5e9bcab2e3 · quinnj/JSON3.jl · GitHub When you run julia with --check-bounds=no, this line is elided and the buffer is not reallocated (by the @check macro).

Ref: Remove inappropriate `@boundscheck` by barucden · Pull Request #311 · quinnj/JSON3.jl · GitHub

nhz2 · April 30, 2025, 1:50pm

This is a great example of why combining --check-bounds=no with third-party packages is a really bad idea. Packages rely on bounds checking for correctness because it is a standard feature of Julia. I personally have never even tested if the packages I work on work with --check-bounds=no.

Topic		Replies	Views
[ANN] JSON3.jl 1.0 Release + new StructTypes.jl Package Data	1	1473	September 25, 2020
Troubleshooting Precompilation Failure of the NMoons Package in Julia General Usage package , precompilation	0	84	April 4, 2024
JSON3 Package: ArgumentError: invalid JSON at byte .... : InvalidChar? New to Julia json	1	1768	September 21, 2020
Local package precompilation fails for Julia 1.9.0 General Usage	4	726	May 14, 2023
[ANN] KeywordStrings.jl --- A fun and convenient string macro for interpolations Package Announcements	1	282	January 16, 2023

JSON3 fails to precompile due to embedded NULs in C strings

Related topics