List of most desired features for Julia v1.x

@mauro3 For sure.

Hopefully this gets integrated into the default Juno setup prior to 1.0, so that users evaluating Julia get a good impression right out of the box. There doesn’t need to be a long-term solution to this issue. I am pretty sure that casual users evaluating Julia would find their way to documentation describing how to tweak performance at the point they need it, but I doubt that people would get far enough to setup and properly use Revise.jl on their own.

2 Likes

I’d definitely like to make Revise “default” for most people; the worry would be causing some kind of regression. I think there are a few things that need to improve before that can be possible, but https://github.com/JuliaLang/julia/pull/23898 goes most of the way. I may add https://github.com/JuliaLang/julia/issues/23448 because one negative of Revise is that it increases load time since it has to re-parse the source code for all that packages you use.

The other concern is that for a few people, file polling seems flaky. https://github.com/timholy/Revise.jl/issues/42. Not sure what to do about that.

By and large, though, if the things on my list get done I think we should just make it part of the default experience.

4 Likes

A graphical debugger, please.

I was a previous Windows user, then switched to Linux and never looked back. But I really miss debugging programs in Visual Studio. If we ever got a debugging experience like that with Julia, in Linux …

3 Likes

I disagree on this for science, as in your code as a [Windows] .exe (machine code).

EDIT: I see now (having read the whole thread…phew), the rest of my post is mostly redundant with what others have said.*

a) It may not be enough, in theory you would need to know e.g. OS/Windows version. Every detail and down to exact CPU version (see e.g. famous [old] Intel Pentium FDIV bug) can affect results (and e.g. environment variables for threading, I believe also in the .exe scenario).

For “reproachable research” I think it’s not just needed to have the same exact data and same source code, you need to know also exact version number of Julia runtime (and dependencies used, unless statically linked in) and the version/type of OS.

Having the source code of your research, not just a binary, helps in validating if there are bugs in your code, and possibly in the Julia runtime, and if you only get an .exe made with a specific Julia runtime, then you force your user to use a possible say 0.6 bug and the user can’t just upgrade to 1.0 and compare.

I’m not saying a .exe can’t be simpler and useful, just less so for science. I just find it very much needed that science users can see the source code. Maybe a compromise solution is possible, where the .exe is a self-extracting archive of the source code and a specific version of the Julia runtime, and it runs your code, but you could with a switch just get the source code.

b) a problem with a .exe is also it forces your users to use Windows…

Possibly a Freudian slip for: “I know the professor is NOT going to bother downloading” Julia. :slight_smile:


*Seems Docker is the “compromise” solution I mentioned above, ideal over regular .exe?:

This is a topic I’ve been thinking about for some time but haven’t yet started a module for. It’s probably deserving of its own discussion separate from this so I’ll keep it short.

My idea has been to embed Julia version information, package versions, and possibly even code in images created in my plots. JPG images allow additional metadata and this could be a good place to store all of the data I mentioned. At some point I’ll get around to testing.

2 Likes

My big 2 are:

  1. Self contained julia projects (Pkg3 I gather will do this).
  2. More mature and user friendly shared memory threaded parallelism.

Yes, I noticed the title changed after your post from “… v1.0”, I think.

I tried pressing “Summarize This Topic”, but it seems not working, maybe a permission issue for me? If anyone can, explaining 1.0 (for breaking changes) vs. 1.x, meaning 1.1+ (for most ideas in the thread, that are not needed and need not strictly make it for 1.0) might be in order.

And possibly list some non-breaking/package examples that people still want to get into 1.0, e.g. debugger…


Listing some examples (not that I want them in the summary…):

Did you mean for 1.0, can this be non-breaking?

0-dimensional Array{Float64,0}:
1.0

For some relevant types or these ones as an example, you would want Array{Union{Float64,Int32},0}. That would keep type-stability (just not sure the compiler could (as an optimization) or should give it for that exact code).

[Note it’s not really useful here to get a Union as every Int32 (but not Int64) is a subset of Float64, just discussion the general principle to avoiding type-instability (that can be useful to have). Also let’s say you go with Unions, it might be a problem to get ever bigger Unions… if you do similar to this on Union values…]

The Union infrastructure (made for Nulls), would also allow you to give you Union{Int32, Float64} ? Since for Nulls at least, it’s implemented by an array of your values first, and then an array indicating Nullness, I’m not sure this would work for scalars; or would give two trivial arrays of one value each… I guess both the values values from each “array” could be optimized into two CPU registers, but not sure; can an array be “bitstype”?

This works:

julia> x=now > 2018 ? Union{Int32, Float64} : Float64
Union{Float64,Int32}

so I guess this is just an error that could be gotten around:

julia> x=ones(now > 2018 ? Union{Int32, Float64} : Float64)
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Union{Float64,Int32}
This may have arisen from a call to the constructor Union{Float64,Int32}(...),
since type constructors fall back to convert methods.
 in one(::Type{Union{Float64,Int32}}) at ./number.jl:64
 in ones(::Type{T}) at ./array.jl:169

This sounds like something you could make a package for, it’s not something specific to the Julia base.

A simple desire: line continuation syntax. Sometimes, I just want to say, “I’m not done yet. Please continue to the next line, even though this line taken by itself appears to be a syntactically correct expression.” Maybe something like ↵?

the_first_transformation_matrix, a_vector_of_singular_values, the_second_transformation_matrix ↵
    = svd(the_original_matrix)

(Yes, I can just move the = up for this particular example, but I’m going for the general case here.)

5 Likes

|> to work with broadcasting

so I can do

x |> fn1. |> fn2. |> fn3.
instead of

fn3(fn2(fn1(x)))

x .|> fn1 .|> fn2 .|> fn3? It works like any other operator like *.

2 Likes

ah… of course!!!

1 Like

Even better if it’s also allowed to put the continuation symbol at the beginning of the second line instead of at the end of the first. This saves a character on the line that is too long anyway. Or multiple characters if using a longer continuation symbol. A three-character symbol and a space could replace four spaces of indentation. (Advanced editors could render the continuation symbol nearly the same color as the background, so that it resembles indentation.)

For the general case: if you end the line with an unfinished expression, Julia will read the next one to complete the expression. Eg

1 +
    2

function lotsofargs(arg1,
                     arg2)
end

If only that metadata carried through to the publication. It won’t. So the only point where it would be useful is when you use the original files. You, or someone you sent it to, or if you host it on a server you have explicit control over.
But I looooove this idea…

Of course, but often I find an old plot and have no idea of the context or I want one specific plot from a folder of countless plots that all look relatively similar. Images can be extracted from PDFs so the metadata might still survive even in digital publication.

I have already done a little bit of work testing how to read it which turned out to be quite simple. It relies on exiftool which I see you have experience with.

This is off the main topic though so let’s move further conversation to a new thread. I’ll post something at some point today once I’ve built a full example. I’m still a bit uncertain of how much is possible or the best approach.

Edit: Added link to the new thread

1 Like

Some way to get guaranteed allocation-free immutable types containing references (especially arrays of such eltype must not be pointer-arrays). I don’t need a pretty syntax for such type-definitions, but I need a way of specifying, unambiguously, the resulting data layout.

In other words I want a guaranteed (not sometimes maybe optimized) zero-cost abstraction to bundle reference-types together. Currently, the only available zero-cost abstractions for bundling values together into a type are for bitstype.

This would solve, e.g., allocating array views, or the token/semitoken API for trees in datastructures.jl.

Such zero-cost types need to be available and the resulting data layout needs to be well-defined, but it does not need to be syntactically pretty (e.g. new keyword: bitstype_like / leaftype, requiring users to specify explicitly, for every member field, whether it is supposed to be inplace or as reference, and the partial order of “bitstype_like contains inplace member” must be non-circular, else compile-time error).

Cf https://github.com/JuliaLang/julia/pull/18632 .

3 Likes

Maybe this is a no-go, or discussed elsewhere, but if my understanding is correct all memory allocated in Julia is implicitly pinned, to make interfacing with external code easier and less error prone I think. For any future performance work on allocation and GC this imposes some pretty significant constraints on the implementation. Is there any willingness to reconsider this? It feels like the sort of semantics you don’t want to break after a 1.0 release.

I’m not sure I understand what you mean by ‘observable’. In general, settling design and syntax issues for 1.0 that will be hard to change later seems most important. Multi-threading may not have much to do with this. But, convenient, stable multi-threading is an essential for many projects. I’d have a hard time selling Julia as “reasonably mature” if it can’t do multi-threading. (Of course, I’m not offering to try to fix it right now. And it won’t kill Julia if it is not there.)

Multi-threading by default, please no. At least, not MATLAB style. I repeatedly see things like users unwittingly starting thread-pools in a loop to invert 10x10 matrices. They move the code to a server and half the CPU time is in system calls. It’s hard to explain; someone has to be an enforcer, because people ignore you if they can, etc. Happens with numpy, too.

Thanks @Tamas_Papp. I know, but I still mess this up all the time. I use whitespace liberally to convey meaning, and I don’t always want to reformat my lines. For instance, we write math like:

  a
+ b
----
= c

so I like to write out operations like:

result = ↵
  this ↵
+ that

or nested with operations lined up, like:

result = ↵
    thing ↵
  * this ↵
+ that

Plus, I think the ↵ symbol seems like an unclaimed and elegant solution. Ok, this isn’t the highest thing on my wishlist, but it seemed easy and helpful in addressing one of the common mistakes.