I want to add a my personal list of desired features. I don’t restrict to 1.0, maybe some are 2.0+.
Most ideas were evoked by the arguments in discourse #5278 and the links therein.
Type system
Keep the type system simple and elegant as it is now. Types and inheritance should remain the only relevant inputs for method dispatching.
Abstract types (stateless, no instances) and concrete types (structs) (optionally statefull, instanciable) shall be the only “type-like” constructs of the language. Other related features (known as traits, interfaces, protocols,…) could be added as optional properties to the types. I will go into details later.
Allow method signatures. That could be parametric subtypes of Function. e.g. Function{Int, AbstractVector{T}} where T.... Used for dispatching: function f(g::Function{Int}) .... The signature does (not) include function name?
Interfaces/protocols
Optionally add method signatures as members of abstract type and struct.
Syntax e.g. struct S ... fname::Function{Int} end.
Semantics: check after end of compilation the existence of specified methods.
Warnings, if missing methods (signature including name).
Abstract multiple inheritance.
Syntax e.g. struct N <:A<:B ... or abstract type N <:A::B::C … where A,B,C are already defined types.
Semantics: For the purpose of dispatching replace chain of supertypes by linearized list of set of all supertypes. This concept is stolen from Scala, see also here: linearization.
Allow singular inheritance from concrete types. That means, new concrete types can be defined by extending existing concrete types, if they do not add own state.
Syntax e.g.: struct N <:S[<:A...] < end where S is has state, N has no own data fields, A additional abstract.
Semantics: The new type shares state and layout with the old one (methods of course). Implicitly define default conversion function N(::S).
Example: struct RichString <: String; blow_up::Function{RichString} end. Conversion: RichString("abc").
Implement implicit conversions. In the case of concrete inheritance (5.), in a restricted context treat all objects x::S as N(x). That would, together with (5.), cover an essential property of traits. discourse
Example: blow_up("def")
Implement automatic delegation to methods of data members.
Syntax e.g. struct S ...; delegate data::T ... end.
Semantics: add all methods of T (known by interface definitions in T) to the method list of S and use them at runtime. Resolve conflicts by defining breadth/depth-first search. This contributes also this old problem: type mixin #816
Type stability
Improve support for type stabilty.
make the {Core.Inference|Base}.return_type[s] methods first class (documented, inBase).
make the output more exact. For example return_type(cmp, (Real, Real)) should be Int64 and not Any.
The feature that I would most want for Julia 1.0 is that anything that starts with _future_ would be a reserved word, i.e. not allowed as a variable name, etc.
This means that if Julia 2.0 introduces a new reserved word x, then Julia 1.x can make _future_x work the same way, without breaking code.
(The @compat macro would turn x into _future_x for those who want to write 2.0 code that is backwards compatible.)
Exactly this. No matter what you do for your 1.0 release, it will be best that there should be a 0.7 release that allows package developers to synchronize with the new language changes.
This will ensure that a 1.0 release really is language stable, with no breaking changes.
For new users, they will want to see that the language is stable and that they can actually use it with the major packages it offers. The language and packages must be synchronized.
It will be a lot better for the entire julia community if there is a 0.7 grace period that lets packages catch up to the new breaking changes, so that the 1.0 will not need to have all those warnings about what is deprecated.
Whether you want to see it this way or not, 1.0 release will be very symbolic to the rest of the world and they will judge it based on the whole experience of it, which will be much more healthy if the packages are all mature for the 1.0 release by the 0.7 grace release.
I think mine are easy. At least they certainly don’t introduce any breaking changes!
A page in the documentation about pointers, bits, and related functions. E.g., pointer, Ptr, Ref, unsafe_copy!, reinterpret, serialize, etc. The C documentation comes close, but having read through it several times, I still need to try a dozen things before I find something that works, and then I don’t really know why. I see that there are many questions on these things on Discourse as well. (Side question: do people feel that these are in a good state for v1.0? All of the parts of this domain feel disconnected from the other parts.E.g., why can I reinterpret an array, but not a scalar? Must I literally create an array of the scalar so that I can reinterpret it? That feels silly, but as far as I can tell, it’s what I need to do.)
The ability to discover functions that operate on types. In other languages, e.g., you can discover all of the methods of a class. With multiple dispatch, it might be nice to discover all methods that specify a given type (or a super of the type, not including Any) in the inputs. [EDIT: methodswith does this (see below). IDE integration would be nice too.]
I’ve been thinking how one could expose this in an IDE like VS Code, but haven’t really had a good idea. I think if folks have thoughts on that, it would be really great. I still remember my C# days, when I essentially learned a library by typing the instance name, hitting . and seeing all the methods I could use in Intellisense, it was just a really, really efficient way to discover things…
There should be some smooth way to trigger methodswith without actually typing it; causing a dropdown menu to pop up, that you could navigate in and select. Any random key would do, basically, just need to pick a good one. Ctrl+. maybe, or alt+.? Sort of a modified field lookup.
I think the correct thing to do is to enable autocompletion for tuples, i.e. when you type (1,2)+tab is show all methods that takes two Ints as input and selecting one will prepend it to the tuple. You can mix types and values in the tuple and allow for unspecified parameters with Any or … (e.g. (Task, …) call methodswith(Task)). That’s what I do in GtkIDE and it work reasonably well, although I have to say I very rarely use it.
That involves a bit more typing than I would prefer. Being able to trigger on just a variable would be nice. In many OO languages, you just type . after the variable, so some elaboration on that would be better, IMHO.
I hope that threads will be stable by 1.0 and that broadcast will use threads by default. Otherwise I fear that thousands of Matlab users will download Julia 1.0 on release day, type log.(rand(1000000,1)) and then complain all over the internet that Julia is X times slower than Matlab. I don’t think answering “yeah but look how fast Julia runs on a single core” is going to cut it when everyone is drooling over new Threadrippers and Core i9s.
There are a lot of cool ideas here. Do keep in mind that if you don’t see your favorite feature on this list, it means it probably isn’t on anyone’s radar. As I interpret it, the core team is really focused on making sure that the language is sane, elegant, and extensible; that’s a big job, and from a purely technical standpoint it’s the only barrier to the release of 1.0 (since non-breaking features can be added later).
As a consequence, if you want something by 1.0 that’s not on the list, chances are you should be rolling up your sleeves (now) and start making it happen. The threading in broadcasting is a great example: I happen to be working on broadcasting as I write this (https://github.com/JuliaLang/julia/issues/20740, one of the issues blocking the release of 1.0), but I have no plans to experiment with threading, because it’s not necessary for releasing 1.0. But adding threading to broadcasting might be relatively easy (perhaps just a few lines of code), and indeed it might have a major impact. But it does require someone to run benchmarks, tune parameters, make sure nothing breaks, and take responsibility for fixing problems if they crop up.
I know that many of the people participating in this discussion are already major contributors to the Julia ecosystem, but especially for those who are not, this is your chance! It’s really one of the last windows for changing the course of a language that we all hope will be enormously successful. For most of us, there may be only a few times in our entire career where we have the opportunity to make a contribution that could improve the lives of millions of people. This may be one of them. Think about it.
Woot!! Issue #5187 has been reopened and is on the 1.0 list! I’m still too much of a noob to contribute to the core myself, but I can nag the heavyweights into doing it for me! Watch out @tim.holy and others, that means I’m free to start some serious nagging about including broadcast threading in 1.0 too!
I assume that you’d be rather surprised and upset if adding threading here meant that log.(rand(10^6)) had observably different behavior compared to not using threads. If so then this is “just an optimization” which means that we can add it at any point since by definition it should not change the behavior of anyone’s code in any observable way. It may seem unintuitive, but the changes that need to be made for 1.0 are not the ones like this (optimizations or features) but the ones that do change behavior in ways that you can observe.
I understand your point: threaded broadcasts or any other optimization aren’t strictly necessary for the 1.0 release since they don’t change functionality. But the 1.0 release might bring a LOT of new attention, so there’s also a marketing aspect - you don’t get a second chance to make a first impression. Since speed is one of the main selling points of Julia, I think taking some extra time to implement (potentially) order-of-magnitude optimizations for common operations should be seriously considered, especially ones that are already present in competitors. If doing this would take too much time then at the very least the roadmap to the most important optimizations needs to be communicated clearly.
Is it desirable to use threads by default? The broadcast machinery is used in a lot of contexts. If we use broadcast on a small array, we need minimal overhead, and even a simple branch to skip broadcasting if the array is small might be costly relative to the computation time. There are also some not thread safe operations you might broadcast, for example I sometimes do Pkg.add.(["Pk1", "Pk2", ...]).
I know the MATLAB users will be disappointed by the initial performance of log.(rand(10^6)), but if we note this difference early in the “Tips for MATLAB users” part of the documentation they should be fine, especially if the solution becomes something easy like @threads log.(rand(10^6)). Also, MATLAB users wouldn’t know about the . syntax and would try log(rand(10^6)), and then they’d have check the documentation anyway to understand the error message.
I think this is exactly the right way to think about it. More-so, with the current state of Julia tooling, you would be lucky if people got to the point where a MATLAB user was disappointed by the initial performance of log.(rand(10^6)).
As an outsider who reads almost everything on discourse and eagerly wants to push Julia to my students and coauthors, let me give some perspective. The thing to remember about Matlab is: most Matlab hate the language, but put up with it because it has: (1) a very convenient and stable development environment; (2) most of the packages they need pre-installed; (3) fast enough runtime performance. They want a responsive development environment moreso than they want fast execution time (which they could accomplish with Fortran if they really needed it). Also, remember that only a subset of computational tasks for many users are primarily constrained by runtime performance. Who cares if it takes 0.1 vs. 1 second to run?
Here is the worrying scenario for a typical Matlab user when they hear Julia is at 1.0:
They examine Julia with the perspective that there may be quirks in the language that they need to learn (e.g. the @threads example above, that they can’t use global variables, etc.) and that runtime performance is a work in progress, but want to see if the general environment is good enough to start working in.
They look to see what the recommended IDE is and see Juno. The OS is likely Windows, but occasionally for OSX - but not the type of OSX user who uses a terminal!
The try to do something simple and get 20-30 seconds of loading up packages. Hopefully that only happens the first time they use a library and they don’t have to restart frequently, or else they abort the evaluation rapidly.
They write a simple script using some simple libraries, try to set a breakpoint in Juno to inspect things and play in the REPL (within the IDE).
If they need to constantly restart things, have trouble inspecting code with breakpoints, or run into huge package loading times as they are doing normal activities, they get fed-up.
At this point, one of two things would happen: (1) They say that Julia is too much of a pain-in-the-ass to use, no matter how fast it might be when the code is finished, and will wait for another major release and/or marketing event; or (2) they conclude that Julia will always be a niche language for linux-based hackers and is more a Fortran than a matlab replacement. They try Spyder or something like that, and decide Python is fast enough and more dedicated towards tooling for their type of usage.
The good news on this nightmare scenario is that things seem to be focused on the right direction (e.g. decoupling the 0.7 and the 1.0 release, focusing on language stability prior to more enhancements, not worrying about performance that can be optimized later, etc.) Juno is slick and good enough in principle to compete (as long as Gallium is kept up to date). What has been a disaster when I tried it are: (1) package load times; (2) the number of times the REPL needs to be restarted; (3) debugger not always kept up to date. The final worry would be that packages are not stablized and updated prior to a 1.0 release.
So, as an outsider who really wants to push Julia my biggest suggestion is: please, please, don’t call it a 1.0 release until you get the Juno, REPL, and Gallium worflow to feel responsive and stable. This may be Julia’s one big shot to halt the python momentum. The 1.0 release is a marketing event, whether you want it or not.
I think this is key. All of these packages have good ideas, but there is no need to have all of them. Two solutions going forward are (i) focus on one package and (ii) have something like Plots.jl but for data tables (this would definitely not be the “right” solution IMO).
Working with data is one of the things most users will have in common, and not having a clear package to help is going to hurt adoption. Like Chris, I am lost as to which one I should use, and (more importantly), which one I should talk about / build on in other packages.
