Get it together, guys!

I think it also boils down to what you consider a programming language to be. Many languages don’t have a REPL, does that mean that Julia with its sophisticated REPL is a product ? Some languages are almost inseparable from their IDE, or come with 3D graphics, or audio backed in.

Personally if I had to design a language for scientific programming, I would definitively consider having graphics built-in (like arrays, or bignums are built-in), since plots are essentials for scientific programming. Having graphics from the start would probably make things easier on the long run, as the community could built upon a common foundation over time. That said I don’t think it would necessarily be a good idea for Julia now, it’s something that has to be done early one.

Ok, but are they essential for a scientific programming language? Having “official” built-in plots, seems counterproductive, it would discourage development on other plotting packages. Julia currently has big problems with plotting because of compile times, but it still has a pretty nice set of plotting packages avaialable. I don’t see how the situation would be better if there were plotting in stdlib.

A big part of what I was trying to get across is that programming languages are basically modular, and that this is a great virtue. A package may come a long, become very popular, but later get superceded by something else. This shouldn’t have to affect the development of the core language. This is one of the reasons why there are plans to make the stdlib have separate version control from base Julia.

8 Likes

After reading the summary in the link, I’d say, scratch R then. But “at least not Python” seems reasonable. (I really didn’t know much about R anyway, I must admit.)

Other people have articulated most of my reactions when reading this thread. I’ll just note that comments like this

while probably written with benign intent, are actually pretty insulting. It made me angry on behalf of all the people (I do not include myself) who are putting an enormous amount of thought and hard work into the language and its ecosystem. I am further angered on behalf of myself, since I use this language every day for work that I think incredibly important - by extension you’re saying that I am irrelevant.

You’re of course entitled to your opinion, but it’s pretty presumptuous to think you know what’s best for this language and community that you’ve spent perhaps tens of hours interacting with, when most of the people responding to you have spent hundreds or thousands of hours on it. Granted, I am probably reading these statements with less charity than I might otherwise because of my reaction above.

:100: :100: :100:

22 Likes

Counterpoint: while Jeff and I both have some background in visualization software, if we had done this then we would have been on the hook for building and maintaining the built-in plotting stuff, which I’m 100% sure we would have done a terrible job of—because building and maintaining the language, its compiler, its package manager, etc. is already a massive amount of work and we do not have time, even if we might have the ability, to also make a good plotting and visualization system. We would have been left with a crappy plotting system that was baked into the language which lacked maintenance and was seldom updated or improved because it was hobbled by the relatively slow release cycle and the very strict compatibility policy of things that are built into the language. So from my perspective you are wishing for a shitty built-in plotting system with no alternative.

Instead, we have a vibrant ecosystem of plotting tools with multiple choices—all of which are excellent and improving as they compete with each other for mindshare and copy each other’s strengths while abandoning their own weaknesses. If you think it would be any other way with a single standard built-in plotting system, you are probably misled by the examples of proprietary systems like Matlab and Mathematica which have the benefit of people paying money just to use them. Money which can be used to pay professional UI engineers and designers to make sure the built in plotting system is good. Julia doesn’t have that luxury (nor do I wish it did—being open source was one of the major points of creating Julia), but it does have the benefit of a large community of people who are both more willing and more able than the creators to make amazing visualization tools. The only drawback is that you as a user need to pick one and then type ] add ChosenPlottingPackage in order to be able to use it.

39 Likes

I don’t care much for this post nor the OP. I’ve been using Julia since 0.5 days and am incredibly happy to see it evolve into what it is today. It takes a lot of time and energy to perfect something. we’ve seen great features get added and usability concerns addressed. No one claims the language is perfect and no one is forcing you to switch. For plotting I usually just switch to Vegalite.jl or export data to ggplot. I still have to get my head around the new package manager. But I know (because I am somewhat active) that the developers know about these issues and eventually address them. Maybe I should actually learn the internals and start contributing if I really want these things fixed.

6 Likes

I am a bit puzzled by the OP because although I am glad that the debugger is available in Julia I do not use it for my own developments. I sometimes (rarely) find debuggers useful when I explore others code but it clearly not one of my priority… So complains about the syntax to use the available debugger makes me wonder about my normality… The wonderful Julia tools for exploring the performance issues (Benchmark Tools and @code_xxx) makes me consider Julia ecosystem much more complete than every other language I have used in the past :wink:

4 Likes

Oh god please, not that again :-p

4 Likes

Nay. All languages for which a REPL makes sense (ie you can compile/execute on the fly and develop interactively) have one. Not having a REPL is something that requires a serious excuse these days.

Arguably Julia had graphics from the start, almost: some plotting packages go back to 0.4 or earlier. It was just not “built in”.

It would have been eminently possible to build in something not unlike the default graphics facilities of R or Octave, and tie it to a release cycle of something totally unrelated (ie the language proper).

I am glad no one did this, because a lot of effort would have gone into something with ultimately the same fate: a default that remains available but in practice happens to be superseded.

4 Likes

I’m thinking more of low levels graphics that the community could build upon rather than full blown plotting library. One issue with the current plotting ecosystem is that the community has to manage binary dependencies, which very regularly breaks (more or less depending on which you use). Maybe I’m wrong but it seems to me it would be easier to ship it directly with Julia, since you already have a cross-platform building and distribution system.

Note that you can also apply your argument to the package manager itself (since in some languages the package manager is not built-in). At the end it boils down to what you think is more important.

I think there’s also a lot of value in having even a minimal, core official system, it serves as a flag the community can rally and build around. It seems the decentralized system works well for some things but not so much for others, where efforts get diluted. Sometimes it’s better to follow a bad plan together.

1 Like

Posting in an epic thread.

35 Likes

I was writing a longer post, but I’ll try to cut it down by focusing on what @Brad_Hines seems to mean by “usability.” So far I was able to glean the following from his posts (and feel free to correct me if I’m wrong here, Brad):

  • It would be nice if JuliaPro were more like MATLAB (or Anaconda, I’ll add)
  • Debugger doesn’t seem to work with Juno on latest JuliaPro and this is rather frustrating

I would like to share my own experience with the debugger because I think it may add something to the discussion here. I have only ever used the Debugger in the terminal/REPL, where I think it really shines. For me, it was really as simple as ] add Debugger; @enter foo() and it just worked immediately. One time it didn’t work when I needed it to because of an underlying issue in JuliaInterpreter which I filed here and was pleasantly surprised when Kristoffer Carlson just went ahead and fixed it a few weeks later.

I bring this up because I think it illustrates an important point: Julia is really great for many things right now, especially for those who are comfortable in a terminal and don’t need a full-blown IDE. I do fine with VIM and a REPL. Even when I used to use VS code I used it as the editor but I ran all my code in the REPL with heavy use of Revise. I think “usability” is often conflated with “has an IDE that does everything I want,” which is not where Julia is right now. Brad Hines is sure not the only person who has made posts here wishing for such an IDE though! For better or for worse, such an IDE probably won’t happen unless the current funding situation changes.

On a final note, it has been much easier in my experience to install base Julia and install packages I need rather than use JuliaPro. I would recommend that to new users. I think package management in general is easier in Julia than in Python; at least I actually understand how to use Julia environments while I can’t say the same for virtualenvs. (I’m really lazy though, so take that with a grain of salt.)

Edit: I don’t want to disparage Juno and VSCode. Those plugins are a great piece of work. Lots of people use them to great effect. I just don’t think they’re quite the same as they would be if their development were funded by a large company to the tune of hundreds of thousands of dollars a year, which is what seems to be a common expectation for IDEs. Hopefully that doesn’t offend anyone?

5 Likes

There’s nothing magical about graphics being built-in. You still have all the same installation problems that you have for getting graphics working as packages. Except that now you’re expanding the problem of “help, my plotting is broken” to “help, my Julia is broken”. If we had the solution (we actually do, see below) to installing graphics libraries and having them just work, that solution would be just as applicable to graphics as add-on as to graphics as built-in. The only way the proprietary systems get around this is by shipping the entire graphics stack as a single application. That’s doable but… it’s expensive to develop, expensive to maintain, it tends to age very poorly as the OS capability races ahead of the proprietary stack.

Note that you can also apply your argument to the package manager itself (since in many languages the package manager is not built-in). At the end it boils down to what you think is more important.

Yeah, how’s that going for them? :joy_cat:

There’s a major difference between building in graphics versus a package manager. You either have a closed system that’s complete and cannot be extended (like Mathematica) or at some point you need a package system. Having graphics built in solves only the problem of installing graphics libraries. It does nothing if you need to install a GPU driver or a MILP solver—or any other kind of binary dependency. But if you get the package manager right, then you get graphics libraries for free. And GPU drivers. And MILP solvers. And everything else. That’s why you want to have a built-in package manager and really try to get it right. Otherwise you end up in quite a mess.

We’re not there yet, but we’re actually working on the solution to installing arbitrary binary artifacts, including platform-specific libraries, without needing any build step. It’s been a huge, many-year project, but it’s almost ready. The package manager side is merged on the 1.3 release branch; the binary builder side is in progress right now. Instead of solving one specific binary dependency problem by shipping a baked in graphics application stack that will age poorly and be hard to maintain, we’re solving the general problem that fixes all the other problems: installing platform-specific binary dependencies. Once you solve that, the headaches of installing graphics packages go away.

36 Likes

And may the question be allowed: “from where?”. Where are these up-to-date, maintained platform-specific binary dependencies waiting to be installed from?

From the mythical Tree of Life, of course:

https://github.com/JuliaPackaging/Yggdrasil

14 Likes

I really like your sense of humor.
I click into C, but somehow i don’t see libcairo, i don’t see Skia, OpenGL, Gtk, Vulkan, Qt.
What i’m doing wrong?

Until a builder is mature it tends to develop outside of the tree, e.g.:

Other ones have been worked on elsewhere but I can’t find them. Some haven’t been built yet.

3 Likes

Sorry, i’m confused now. Upthread you mentioned, that julia+Pkg solves the problem and now suddenly the maintained binaries ‘have not been built yet’. What is the time line estimation here, that they exist? And who will put effort in that?

I’m not sure how “they can be built” requires “they have been built”. There will be plenty of information about this forthcoming, I don’t feel like getting into it (more) now. You can watch Elliot’s video from JuliaCon 2018 for more info on how building binaries for lots of different platforms uniformly has been addressed:

10 Likes

Counterpoint: I’m one of two people in a startup company and we made the choice from the get-go to use Julia to develop our product due to its combination of performance + ease of use. We started out initially post-1.0 but pre-debugger. I had some experience with Julia in grad school, but I came from a primarily Matlab-based research group and I had worked in Python at my last job. Obviously, we had to adapt our existing programming styles to fit the style- & performance-requirements of Julia and were using a ton of REPL-based development from the outset. Fast forward to today, where we write highly performant Julia code, use Debugger.jl for development + debugging, manage our own registry of private Julia packages, and have a full Docker-ized development + Gitlab-based CICD testing framework. It took a while get all these things up and running from scratch because I was the only person setting it up (and that isn’t my background), but honestly most of that time was spent in the setting up of the dev-ops tools + environment relative to dealing with problems with Julia. There are a lot of resources out there to amalgamate all of various software components together into a cohesive development stack (praise Google), but it is very feasible (and dare I say, enjoyable) to use Julia in a production setting.

25 Likes