This morning I did a standard upgrade to Julia 1.6.2 and loaded the basic packages I use for Data Science. After adding everything, I tried to get started, beginning with the simple statement ‘using CSV’ and got a page of (apparently unresolveable) package dependency issues. Fact: Julia 1.6.2 is not (reliably) compatible with any version of the package CSV apparently at the moment (a known issue - thread on Discourse) What I want to know is how such basic functionality as csv-reading/writing can become broken in a standard/default installation of Julia at this late stage of it’s development!!! I am posting on here because this seems to be more than just a minor software bug, but the kind of systemic failure (if I may say) that threatens the growth and acceptance of Julia itself, and (speaking as an academic) makes it very hard for me to make a case for using Julia in my teaching and research, which (given Julia’s obvious potential) I see as a terrible waste. Apologies, but I could not let this go without noting

Thank you for mentioning that you observe this in Jupyter, that clears it up! IJulia itself depends on JSON, which brings in the Parsers problem that’s been observed in the other threads. Since in this case (as @mbauman mentions) the dependencies for the IDE (IJulia/Jupyter) are not seperated from the regular code, you get the same kind of error as observable in Atom (which has the exact same problem, as I understand it). Unfortunately, there’s no one party that could see this coming - if anything, it’s a legacy problem stemming from the approach both Atom and Jupyter take for loading code, not directly a problem of Parsers releasing a new version. If I’m not mistaken, the solution should be to pin the package (]pin Parsers@v1.1.2, possibly followed by ]up, not 100% sure) in whichever environment you’re starting the IJulia kernel from (before using IJulia or starting the jupyter kernel).

[image] compleat: I am not talking about an esoteric, advanced feature here, but the ability of reading and writing data, which should be sacrosanct in any software package like Julia I just wanted to add that Julia is not a software package. It’s a language with standard libraries, which one could count as a software package, yes, but most functionality comes in third party packages. There is just not really a “governing body” that might have messed up here or whose priorities need to be straightened out because they don’t take csv import seriously. It’s simply collateral damage of the normal process of updating and versioning disparate pieces of software. Note that if you’re at the forefront of versions, you’re bound to have issues like this once in a while. That’s just due to the complexity of the ecosystem, especially because the Julia ecosystem has so much code reuse and composition, where one tiny issue in a dependency can have ripple effects. If you use Julia for courses, you can’t go wrong in setting up environments with known good versions, then distributing them to all your students. There might come a time when there can be made no more substantial improvements to CSV and DataFrames, so people won’t have to care anymore whether they’re at the bleeding edge or not, but this time is not here, yet.

OK, thanks for your thoughts. I guess I am assuming that there is a community that would like to see Julia prosper and grow, giving my thoughts from that point of view. [BTW, I did manage to fix things with CSV by fiddling with some of the comments above.]

Please don’t be overly incendiary. This is a very engaged community who wants to see Julia prosper and grow, and you know this. You’ve previously asked how to gear your courses for success and gotten lots of engagement — and lots of suggestions on how to avoid surprises by distributing a stable and known-good environment of specific package versions to your students. I know we can continue to do better, but that’s a really good way of making sure things like this don’t happen to you or your students in the first place. Very glad to hear you’ve resolved this!

[image] compleat: I think one can’t use DataFrames without CSV Strictly speaking that is not what the DataFrames docs say.

[image] mbauman: Please don’t be overly incendiary. I always post with respect for what Julia and the Community is. That said, I do like to convey a sense of urgency when I see it, and I know from experience that in any venture, getting the basics right for the simple things is a “make or break” matter. I want to see Julia prosper and grow, but I know that will not happen if bugs like this (just simplest reading/writing, standard/common configuration) persist. I have expended quite a bit of energy trying to argue the case for Julia in my institution (with some success), but when students get frustrated by a fundamental unreliability of something very basic like this, it wholly undermines my case for Julia being a sufficiently mature language to adopt, and (in my opinion) at this stage, I find this somewhat alarming, to say the least. In my opinion there has to be priority placed on the reliability of the basics, no matter how elegant the project wants to ultimately be, or it will have very little chance of succeeding in the long-run.

Another workaround (which I actually like as the main developing workflow for jupyter notebooks) is to use the Jupyter extension inside VSCode, since it does not rely on IJulia, it is able to downgrade Parsers and it works. You can open your notebooks there. [image]

That sounds like a neat idea. I wasn’t aware until this issue arose that IJulia could break dependencies. It also hadn’t occurred to me that the VSCode approach to Jupyter didn’t require IJulia. Thanks.

As many folks mentioned already, we generally don’t have these problems in the Julia extension for VS Code, because we load all the packages that the extension needs to function in a non-standard way, so that user code can then load different versions of the same packages and they can co-exist. That in principle is also true for the new native Jupyter Notebook feature we have in the extension, BUT, I just give a fair warning that this notebook feature in VS Code for Julia is still a bit experimental and new. Because we don’t think we have ironed out all the corner cases yet, we do not enable the feature by default, instead you need to change a configuration setting to get access to the native Jupyter Notebook feature in the Julia extension. I had originally hoped that we might be able to move out of the experimental mode very soon, but right now it looks like it might take another cycle of upstream releases by VS Code itself (we are waiting for some APIs from them to finish the implementation).

Oh, and one other point: there is also CSVFiles.jl. We use that extensively in my research group and orbit. The package moves relatively slowly in terms of new features etc, for us that kind of stability is important, so that might be another option to try.

Issues with CSV in Julia 1.6.2

Specific Domains Data

Topic		Replies	Views
CSV.jl fails precompling, TypeError: in Type{...} expression, expected UnionAll, got Type{Parsers.Options} General Usage question , csv	19	3983	December 8, 2021
Precompiling CSV.jl fails General Usage package , error , csv	13	3618	May 9, 2023
Reverting back to avoid precompile errors? New to Julia question	26	2450	September 1, 2021
Julia CSV.read stopped working Data csv	32	3462	April 30, 2022
CSV.read not recognizing "select" keyword Data csv , ijulia	11	1088	June 6, 2022

Issues with CSV in Julia 1.6.2

Related topics