Failed to precompile CSV in Pluto...?

Tooling Pluto
,
1

Just started a fresh project environment this morning and CSV won’t compile when calling using CSV from within Pluto. It works just fine when calling from REPL, the error only occurs when using Pluto.

The error in my notebook is:

Failed to precompile CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b] to C:\Users\mthel\.julia\compiled\v1.5\CSV\HHBkp_tAWrN.ji.

error(::String)@error.jl:33
compilecache(::Base.PkgId, ::String)@loading.jl:1305
_require(::Base.PkgId)@loading.jl:1030
require(::Base.PkgId)@loading.jl:928
require(::Module, ::Symbol)@loading.jl:923
top-level scope@Local: 1

And then the REPL looks like this:

      From worker 2:    ERROR: LoadError: UndefVarError: invrefpool not defined
      From worker 2:    Stacktrace:
      From worker 2:     [1] getproperty(::Module, ::Symbol) at .\Base.jl:26
      From worker 2:     [2] top-level scope at C:\Users\mthel\.julia\packages\PooledArrays\4I2sD\src\PooledArrays.jl:162  
      From worker 2:     [3] include(::Function, ::Module, ::String) at .\Base.jl:380
      From worker 2:     [4] include(::Module, ::String) at .\Base.jl:368
      From worker 2:     [5] top-level scope at none:2
      From worker 2:     [6] eval at .\boot.jl:331 [inlined]
      From worker 2:     [7] eval(::Expr) at .\client.jl:467
      From worker 2:     [8] top-level scope at .\none:3
      From worker 2:    in expression starting at C:\Users\mthel\.julia\packages\PooledArrays\4I2sD\src\PooledArrays.jl:162
      From worker 2:    ERROR: LoadError: Failed to precompile PooledArrays [2dfb63ee-cc39-5dd5-95bd-886bf059d720] to C:\Users\mthel\.julia\compiled\v1.5\PooledArrays\vi11X_tAWrN.ji.
      From worker 2:    Stacktrace:
      From worker 2:     [1] error(::String) at .\error.jl:33
      From worker 2:     [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1305
      From worker 2:     [3] _require(::Base.PkgId) at .\loading.jl:1030
      From worker 2:     [4] require(::Base.PkgId) at .\loading.jl:928
      From worker 2:     [5] require(::Module, ::Symbol) at .\loading.jl:923
      From worker 2:     [6] include(::Function, ::Module, ::String) at .\Base.jl:380
      From worker 2:     [7] include(::Module, ::String) at .\Base.jl:368
      From worker 2:     [8] top-level scope at none:2
      From worker 2:     [9] eval at .\boot.jl:331 [inlined]
      From worker 2:     [10] eval(::Expr) at .\client.jl:467
      From worker 2:     [11] top-level scope at .\none:3
      From worker 2:    in expression starting at C:\Users\mthel\.julia\packages\CSV\YUbbG\src\CSV.jl:15

So the problem appears to be with PooledArrays. Is this something wrong with setup on my end or should I file an issue?

Line 162 from PooledArrays.jl is DataAPI.invrefpool(pa::PooledArray) = pa.invpool.

1 Like
2

I’m getting this same issue on Ubuntu 20.04. Hasn’t happened before today. Here are my Pluto and PooledArray versions (latest at the time of posting):

(@v1.5) pkg> status Pluto PooledArrays
Status `~/.julia/environments/v1.5/Project.toml`
  [c3e4b0f8] Pluto v0.12.20
  [2dfb63ee] PooledArrays v1.1.0
3

I get a similar issue on Ubuntu 20.10

4

I have a partial workaround! I started downgrading packages and arrived at this configuration that works:

(@v1.5) pkg> status CSV DataFrames PooledArrays CategoricalArrays
Status `~/.julia/environments/v1.5/Project.toml`
  [336ed68f] CSV v0.8.2
  [324d7699] CategoricalArrays v0.9.0
  [a93c6f00] DataFrames v0.22.3
  [2dfb63ee] PooledArrays v0.5.3

(@v1.5) pkg>

I’m not sure exactly where the issue is, and it might be in just one or two of these packages. I’ll try to keep narrowing it down from here. I got to this state with

(@v1.5) pkg> rm CSV
(@v1.5) pkg> rm DataFrames
(@v1.5) pkg> rm PooledArrays
(@v1.5) pkg>  rm CategoricalArrays
(@v1.5) pkg>  add CSV@0.8.2
(@v1.5) pkg>  add CategoricalArrays@0.9.0
(@v1.5) pkg>  add DataFrames@0.22.3
(@v1.5) pkg>  add PooledArrays@0.5.3

Hope this helps!

1 Like
5

Thanks! I opened an issue for this at the Pluto.jl repo since I’m not getting the error in a regular Julia session/REPL. Hopefully it’s a quick fix for @fonsp :slightly_smiling_face:

6

Just to confirm from my side (I discovered this thread only when on issues for Pluto on github). It affects not only CSV.

CSV (0.8.3), DataFramesMeta (0.60) and DataFrames (0.18.3) load & precompile nicely when run from the REPL include("pluto_notebook_file.jl") but all fail to precompile in the Pluto notebook environment.

Since it doesn’t happen to any other packages, well at the moment at least, I wonder if the bug is in Pluto or in these packages (and other packages might follow suit).

1 Like
7

I tried similar, removing CSV, DataFrames and PooledArrays and then added them back again, I did not specify a version but the same versions as yours were added, however it did not improve anything in my case and I had the same Failed to precompile... error in the Pluto notebook environment but not in the REPL.

8

As j-fu said here clearing packages (~/.julia) and reinstalling Pluto fixes the issue. Investigating more!

2 Likes
9

I had the same issue. I solved it by

Pkg.add(PackageSpec(name="DataAPI", version="1.4"))
3 Likes
10

Would it be correct to say that this error, such as below (which also happens in CSV and other components)

[ Info: Precompiling DataFrames [a93c6f00-e57d-5684-b7b6-d8193f3e46c0]
ERROR: LoadError: UndefVarError: invrefpool not defined

is due to significant breaking change in PooledArrays.jl? And so to resolve we need to pin version PooledArrays@0.5.3?

I did it and it worked ok. Everything precompiles nicely.

11

I got CSV precompiled in Pluto first. Then specified CategoricalArrays@0.9.0 following @thatcher’s post and got DataFrames precomplied in Pluto.
It seems that CategoricalArrays@0.9.1 not working with Pluto@0.12.20 either.

12

This is the step that worked me. I did not have to clear anything in my .julia folder.