Reverting back to avoid precompile errors?

New to Julia
1

Hi,
I am doing some work quite unrelated to the reading of CSV-files, however I am very much impacted by the issue described in csv-jl-fails-precompling #67195.

Since my preferred package (GeoArrays.jl, which has a dependancy to CSV.jl) worked earlier this summer, I tried reverting back to Julia 1.5.4, however I still get the same up-to-date-version of CSV.jl and therefore the the issue persists.

Is there a way forward for me (besides just waiting for the CSV-issue to get resolved)? Is there, for example, a way of only loading packages that are older than some chosen date (kind of like a time-capsule)?
Martin

PS. I am in principle using environments, however it appears I didn’t make a proper backup of the appropriate manifest file. I must also admit that I find it a bit stressful to maintain a library of manifest file in order to be prepared for situations like this one.

2

https://pkgdocs.julialang.org/v1/managing-packages/#Pinning-a-package

So e.g.

]pin Parsers@v1.2.1

should do the trick.

2 Likes
3

Hi Sukera,
Thanks for your speedy response. Unfortunatly the work-around in question doesn’t work for me, the problem persists even though I have the following in the manifest file:
[[Parsers]]
deps = [“Dates”]
git-tree-sha1 = “bfd7d8c7fd87f04543810d9cbd3995972236ba1b”
pinned = true
uuid = “69de0a69-1ddd-5017-9359-2bf0b02dc9f0”
version = “1.1.2”

[[CSV]]
deps = [“Dates”, “Mmap”, “Parsers”, “PooledArrays”, “SentinelArrays”, “Tables”, “Unicode”]
git-tree-sha1 = “b83aa3f513be680454437a0eee21001607e5d983”
uuid = “336ed68f-0bac-5ca0-87d4-7b16caf5d00b”
version = “0.8.5”
…

Martin

4

Did you try removing the CSV package and to re-add it to a specific version, e.g., as described here?

1 Like
5

I did. In a clean environment in directory .../testenv, in pkg mode:

(@v1.7) pkg> activate .
(testenv) pkg> add Parsers
(testenv) pkg> pin Parsers@1.1.2
(testenv) pkg> add CSV@0.8.0

Reverting to REPL mode:

using CSV
[ Info: Precompiling CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b]
ERROR: LoadError: TypeError: in Type{...} expression, expected UnionAll, got Type{Parsers.Options}
Stacktrace:
...

CSV@0.8.0 requires Parsers@1.x.x, but the error linked in the first post remains. Pkg.status() gives

(testenv) pkg> st
      Status `...\testenv\Project.toml`
  [336ed68f] CSV v0.8.0
  [69de0a69] Parsers v1.1.2 ⚲

Although I suspect it makes no difference, I’m running Julia-1.7.0-beta4 in Windows10.

6

The versions mentioned in the linked post are

  • Parser@v1.2.1
  • CSV@v0.8.5

whereas you have Parsers@1.1.2 and CSV@0.8.0. Can you try with the versions above instead?

7

For me this configuration works:

(testenv) pkg> st
      Status `~/tmp/testenv/Project.toml`
  [336ed68f] CSV v0.8.5
  [69de0a69] Parsers v1.1.2

julia> using CSV
[ Info: Precompiling CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b]

and also with CSV@0.8.0 with Julia 1.6.1 (and on Ubuntu). The setting mentioned by @Sukera does not work on my machine.

8

Interesting. Maybe @quinnj knows more about why that particular combination doesn’t work?

9

There is no version 1.2.1 (I suspect that was a typo for 1.1.2), but @mike_k seems to have it working with an earlier version of Julia than I’m using. I guess I’ll just have to wait.

1 Like
10

sorry for my impreciseness,… yes, there is no version 1.2.1 (this is what I have meant)

1 Like
11

Thanks so much for your efforts, however neither the combination
Parsers v1.1.2 + CSV v0.8.0 nor Parsers v1.1.2 + CSV v0.8.5
work for me. So I am also in waiting mode.

In the mean time:
Is some kind of time-capsule functionality (as hinted at in the original post) feasible? Or can we (and by that I mean someone much more competent than me) come up with other ideas for how to mitigate situations like this in the future?

Also, for future robustness: Does the whole package need to fail to load, if one part of it (or one dependency) fails to precompile?

12

That functionality already exists: If you keep your old Manifest.toml around and only instantiate it, pinning all versions to their original ones and don’t update, old code will still work as it originally did (provided you’re using the same julia version). Manifest.toml has as its sole purpose to record package versions.

A dependency is a dependency for a reason: The package requires functionality provided by it. Since the smallest unit of encapsulation is a julia package, there is no smaller part that could be reasonably swapped out for something else - it’s a whole version or no version. There’s no mixing between versions of the same package.

If all dependencies are properly versioned, package versions should only be able installable with working versions. If some sub-dependency fails to precompile, something has gone wrong during that version tagging process.

–

I don’t understand though why neither combination works for you. Are you certain those two are the versions actually installed & pinned? Do you have any other packages in those environments? Can you try two empty environments with both combinations of packages?

1 Like
13

For my earlier post,

I started with a clean environment, pinning Parsers to version 1.1.2. I separately tried both the current CSV (0.8.5) which has Parsers v2 its Project.toml and an older version (0.8.0) which has Parsers v1. The first failed immediately, saying there was no compatible version. The second failed on using CSVwith the error I reported above.

14

And what about explicitly asking for CSV@v0.8.5 while having Parsers pinned to 1.1.2?

(@martin.demare please also try all these versions in a clean environment!)

15

Hi Sukera,
Unfortunately it doesn’t work for me.

PS. My project file looks as follows:

name = “mardm_project”
uuid = “f215c453-d48a-4ef3-91ee-74b828a867ac”
authors = [“martin”]
version = “0.1.0”

[deps]
CSV = “336ed68f-0bac-5ca0-87d4-7b16caf5d00b”
Parsers = “69de0a69-1ddd-5017-9359-2bf0b02dc9f0”

And the manifest contains:

[[CSV]]
deps = [“Dates”, “Mmap”, “Parsers”, “PooledArrays”, “SentinelArrays”, “Tables”, “Unicode”]
git-tree-sha1 = “b83aa3f513be680454437a0eee21001607e5d983”
pinned = true
uuid = “336ed68f-0bac-5ca0-87d4-7b16caf5d00b”
version = “0.8.5”

[[Parsers]]
deps = [“Dates”]
git-tree-sha1 = “bfd7d8c7fd87f04543810d9cbd3995972236ba1b”
pinned = true
uuid = “69de0a69-1ddd-5017-9359-2bf0b02dc9f0”
version = “1.1.2”

16

fyi: just to exclude the following: I gave it a try with Julia-1.7.0-beta4 and it still works on my machine (Ubuntu 18.04). Maybe the problem is related to Windows (?). Sorry, I have no more ideas.

17

Actually it seems to work outside Atom (terribly sorry that I didn’t check that until now!). I will repost when I have updated/downgraded the atom related packages to make it work also inside atom (possibly it is just a simple update that is required). Regardless it means that I can work again, so: good times! :slight_smile:

1 Like
18

Same for me. Starting in an empty directory, activating that directory, then adding Parsers and pinning to 1.1.2, any of
add CSV, add CSV@0.8.0, add CSV@0.8.5 does the initial precompilation, but fails with

...expected UnionAll, got Type{Parsers.Options}

when trying using CSV.
I thought I didn’t understand the dependency chain. Now I’m sure of it :frowning_face:

EDIT: I should have said that this is all in the REPL, using Windows Terminal.

19

I can’t repro this on Windows 10 (in Hyper, but I very much doubt that the terminal plays any role here):

(@v1.7) pkg> activate --temp
  Activating new project at `C:\Users\ngudat\AppData\Local\Temp\jl_Lapatm`

(jl_Lapatm) pkg> add Parsers@1.1.2 CSV
    Updating registry at `C:\Users\ngudat\.julia\registries\General.toml`
   Resolving package versions...
   Installed Parsers ────── v1.1.2
   Installed PooledArrays ─ v1.3.0
    Updating `C:\Users\ngudat\AppData\Local\Temp\jl_Lapatm\Project.toml` 
  [336ed68f] + CSV v0.8.5
  [69de0a69] + Parsers v1.1.2
    Updating `C:\Users\ngudat\AppData\Local\Temp\jl_Lapatm\Manifest.toml`
  [336ed68f] + CSV v0.8.5
  [9a962f9c] + DataAPI v1.7.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [69de0a69] + Parsers v1.1.2
  [2dfb63ee] + PooledArrays v1.3.0
  [91c51154] + SentinelArrays v1.3.7
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.5.0
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [de0858da] + Printf
  [9a3f8284] + Random
  [9e88b42a] + Serialization
  [8dfed614] + Test
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [4536629a] + OpenBLAS_jll
  [8e850b90] + libblastrampoline_jll       
Precompiling project...
  4 dependencies successfully precompiled in 11 seconds (8 already precompiled)

julia> using CSV

julia> CSV.read
read (generic function with 2 methods)
20

This made me wonder what might be different between your setup and mine. That suggested I try running Julia without my usual startup file. When I did that, I got exactly the same results that you did.

Then I did a series of runs, repeating what I had just done, but adding one of the packages I usually load in my startup before activate/add/using. Everything ran smoothly until I added JuliaFormatter first. Then I had the compilation error.

It might be that other packages that require some sort of parsing produce similar results (Revise is one I thought could be a problem, but it isn’t). If nothing else, I nowe understand that an environment is not necessarily as ‘clean’ as I think it is.