I saw in a thread: “we have effectively lost the JDBC wrapper until Julia 2.0”.
This isn’t strictly true, as it still works in Julia 1.0.x, the LTS versions for the foreseeable future.
But, yes, Julia 1.1, if I recall, broke JavaCall.jl and thus JDBC and I guess Scala/Spark too. [And for JDBC there are workarounds.]
I’ve promoted Julia’s “syntax stability guarantee”, since Julia 0.7. I do occationally see packages being fixed for 1.1, 1.2 or 1.3. I realize there can be changes in the standard library, requiring fixes in packages. This seems to be rare and soon fixed (and Compat.jl may also help).
I do not think this is a major worry, except possibly in this one very specific JavaCall.jl case. And then only if your other packages need Julia 1.1+.
Is it true the fix is only likely in 2.0, not sooner? Is breaking happening increasingly rarely?
That breakage has nothing to do with Julia’s API stability, it’s due to a bug in how the JVM deals with the stack in the presence of native code using multiple stacks. As a result, the JVM is unhappy with the way Julia manages stacks. It’s possible to patch the JVM so that it doesn’t freak out, and it’s also possible to compile Julia to avoid the issue at a cost of performance (by using only one stack and copying each tasks stack back and forth).
I don’t actually think any additional work has been done on this, so it seems unlikely that any fix will be in 1.3 which is due for feature freeze quite soon.
Probably not very helpful, but this issue is kind of scary in “big data” world. Spark is increasingly ubiquitous and JDBC was a really good fall-back for database interaction. Not having JavaCall.jl will seriously endanger Julia’s usability for a fairly wide audience. I’m not overly fond of Spark myself, but I have had cause to worry about this a bit lately, I’m a little scared to tell people that it’s an ordeal for me to call the JVM.
I wish I could offer help with this issue, but my knowledge falls well short, I don’t think I even understand what the problem is.
As far as I have heard, the release date of 1.2 will also be the feature freeze of 1.3 and it will happen around the 10th of August (~ 4 month after feature freeze of 1.2).
I have to admit that the 1.3-alpha business also confuses me. Having 3 versions discussed at the same time is not optimal. But I guess this is just because 1.3 brings the new threading architecture and people were curious/trying it out already.
My point with asking here, was is there’s a more general problem? This is the only package I recall stuck on Julia 1.0.x (so I don’t think so, except for all packages depending on JavaCll); however people feel it’s a problem for JDBC (and I guess Taro.jl, Spark and more; e.g. promoted at Julia Computing: Julia and Spark, Better Together).
Despite my title, there’s also a problem with compatibility for packages for Julia 0.6 (or earlier), e.g. ROOTS.jl stuck there. I know nothing was promised, and good tools to help upgrade, still people need to do the/some work.
This got me thinking, is it a crazy idea to implement JuliaCall.jl, to help with JDBC (Julia calling older Julia, calling JVM…)? SImilar to Yuri D'Elia / Polyglot.jl · GitLab (or just an extension to it; yes, that package is stuck on Julia 0.6 it seems, but should be an easy update). One of the point of pybond which Polyglot.jl is based on, was Mixing Python 2/3 code bases.
I see pros and cons:
Old code will just work.
No pressure to upgrade old code (a con if you will).
Allows remote, and while slower/inferior (to e.g. PyCall that doesn’t), you don’t always need mixing data in the same process as PyCall does, nor would it work here for Julia-to-Julia? Even if you run Polyglot not remotely, it’s slower, but may not be a problem for e.g. JDBC. Probably not for all code, e.g. ROOTS.jl?
The 1.2 RC process has taken an unusually long time—this happens some times
We wanted 1.3-alpha out so that people can start testing the multithreading work early
Given the amount of time it can take to get through the RC process, we need to move to a model where we are concurrently working on stabilizing one release while also working on features for the next release, so this will become the norm, except that there will not usually be alpha releases. In this case, 1.2.0 final will be released around the same time as we feature freeze 1.3 and create its release branch.
We want package authors to be as strongly encouraged to upgrade to 1.0+ so allowing people to keep using 0.6 packages indefinitely seems like it would be actively damaging to Julia’s ecosystem. Being able to run two different Julia versions in the same process would be a huge amount of work, would have a significant detrimental performance impact, and wouldn’t fix the problem with Java—it would probably make it worse, in fact—all of which makes this not seem like a very appealing plan.
Not what, I meant. Not doing that would be I think trivial work. I was probably editing my comment when you answered, see on adding to Polyglot.jl that doesn’t use the same process.
Sure, if someone can get RPC between different versions of Julia working, that would be possible, but it does not really seem worth the effort. One of the good things about open source though is that whoever thinks something is worth doing can just go ahead and do it.
I’m looking into that. FYI: Indirectly to help me with at, I’m trying to use this package (that must have worked at some point), and I get a segfault when precompiling in Julia 1.0.4, but not in Julia 1.1 (can someone explain the ERROR there?), while still not working… so I opened an issue:
[EDIT: Not sure about 1.0.x, but still should precompiling for “using Py2Cl” segfault? Possibly some missing “1.1” in some files, I’m mostly familiar with REQUIRE. Initial version was released 19 Aug 2018 days after Julia 1.0.0 released then with Manifest.toml Py2Jl.jl has Project.toml etc. and never had REQUIRE, so I read that to be yes to at least 1.1+ (or 1.0.x) but no Julia version, I’ve checked, works.
This only occurs in Julia 1.0.4. About four months ago I used to try debugging julia source code and then it seems that it’s a string comparison in codegen.cpp causing the segmentation fault.
However at that time very people have encountered this, so I think the best option for me is just using Julia 1.1+.
I do have several ways to reproduce this, but it’s really sophisticated and too package specific.
Like via using MLStyle; using DataFrames and then simply doing something with the two packages.
However I think it must be myself who triggered the segmentation fault: Py2Jl doesn’t have DataFrames.jl dependency, but it still produce the segmentation fault.