There’s many kinds of end-users because there’s many kinds of applications. The same package could be used by end-users in some contexts and by developers in others. I just broadly think of an end-user as someone who doesn’t implement most of their function calls in some context, whether it be plotting, audio playback, and, yes, even modeling languages.
I disagree with the notion that something assuming knowledge of jargon means it is not for end-users. By that reasoning, nothing is for end-users, not even applications like games or chatbots, because those very rarely give users a tutorial on context-specific terms. I’m getting a personal lesson in that as I am trying out PC gaming; no game tells me exactly how WASD moves or how to force-quit with Alt-F4. If we try to limit ourselves to contexts where there’s very little jargon that can be easily taught, we’d exclude many popular applications, which defeats the purpose.
I think a marketable strength of Julia is that end-users can write small Julia snippets that the compiler can optimize and integrate into a package’s pure Julia functions. Interactivity + optimization is really just another way of saying “solving the two-language problem”, but it’s worth pointing out how it can be a package’s feature.
That’s not to say I’m against wrappers, though. We don’t need custom code in some contexts, and wrappers are a good way to write Julia in new contexts with standard performance.
Yep. Can we please try to stay civil here? I agree with the thrust of @Sukera’s point, which I took to mean that people who use differential equations all the time are not even close to representative of people who are interested in doing “computer stuff”. The SciML stuff is fantastic, we should all be clear about that, but only until very recently has the Julia community started making tools that are broadly applicable, stuff like Genie or Franklin.
By “canned tools” I suppose I spoke imprecisely. I would love to see more general purpose tooling that falls outside of the numerical computing pigeonhole we find ourselves in.
Can someone give some more examples of the kind of program we’re talking about?
It seems to me that Julia is not the language of choice unless one needs interactivity or numerics. Things like Genie or Franklin are for Julia users who need to make websites – few will pick up Julia because they want to make a website.
We should consider working on I/O. The standard libuv-based I/O is ok for some situations, but can be outperformed in many situations. This is addressable by Julia since calling low-level OS specific I/O functions are a mere one line ccall away.
Builiding on this, one could setup a series of microservices that perform heavy numeric computations. Imagine a Pluto.jl notebook but at enterprise scale. Julia could serve as a backend for Microsoft Power BI dashboards.
Julia based server applications play well to Julia’s strengths and weaknesses. Code loading becomes neglible since this can be amortized over the uptime of the server. Meanwhile, we get fast executable code that can run on heterogenous hardware. Other services can take advantage of Julia’s strengths by connecting to those services.
Eventually, Julia’s wasm runtime capabilities could improve meaning that you could use Julia from backend-to-frontend, perhaps rivaling JavaScript where numeric really matter.
To be clear, I do not think that any of this requires Julia 2.0 at all. However, it may involve additional runtimes for Julia other than the current one.
I think Pluto.jl has a lot to offer for teaching, and @fonsp is putting in a lot of work to improve his Computational Thinking class. To me, that’s a rather stellar example of making Julia accessible and fun. However, it’s not targeted towards “end users” but rather “end learners”.
On the more operational side of things, of course there is SciML, but also Turing.jl (an awesome gateway to probabilistic modeling) and JuMP.jl (an interface to all of mathematical programming), along with a few other large projects.
These are well-known to Julians but not nearly as widely used as they should be outside our beloved bubble. Which raises two questions:
How do we make these awesome ecosystems more accessible to people who want to code? For instance, if you compare the “getting started” tutorials, I feel like the JuMP one does much more beginner handholding than the Turing one or the SciML one.
Maybe what it takes is starting from scratch every time. In the SciML case, that would mean introducing what a differential equation is and why it’s important, before the first example even appears.
How do we make them more useful who people who don’t want to code? That’s an entirely different problem, and it requires wrappers or interactions or websites which provide an intuitive experience. That’s not my specialty, but I’m very curious about the GUI format for ModelingToolkit that @ChrisRackauckas mentioned. That sort of thing could definitely be part of the solution.
Leaving aside the 2.0 issue, I’ll focus on the “what steps” part of this post.
Perhaps the answer is to take part in wider discussions. For example, the webpage introducing an upcoming meeting about spatial analysis (see Spatial Data Science across Languages (SDSL)) has three icons at the top of the page: one for R, one for python, and one for Julia. Folks attending the meeting have an interest in getting their own work done. If they see that Julia offers something they need, then the language will have attracted new users.
From its early days, Julia has had proponents arguing that it is simply the best, better than all the rest (to quote a song). That’s all well and good for the initial stages, to get the word out to a community.
But now it may be a good time to just show Julia in action, in applications that are relevant to a wide range potential users. If an expert in some particular topic shows that Julia offers advantages to others who work in that topic, then no evangelizing will be required.
We have total downloads and unique IPs (or approximately unique users) tracked in the package server. For example, SciMLBase has about 150,000 unique IP accesses and ModelingToolkit has about 50,000 unique IP accesses. This package server is made available to the community (in an anonymized form like this).
I guess I’m very confused about what this audience is supposed to be then. What is the audience of SciML that is too high level to know what a differential equation or linear system is, but too low level to want to use a GUI? SciML has many entry points and most are not for high level end users, I’d say I’m even trying to move DifferentialEquations away from the “public” consciousness and making things more about ModelingToolkit in the first entry because that starts a lot of people in an easier spot than “define your own function” (which means performance is in their hands, not ours). But ModelingToolkit and EasyModelAnalysis is intentionally made to be the simplest high level model description we can think of, and a package for doing the simplest model queries that I can think of. Then the GUIs (cough JuliaCon cough) are the next higher level that we’re working on to make it so that non-coders can access the functionality.
I’m very confused as to what this supposed intended audience that is in the middle is, and what the right tools and documentation would be focused on them. Who is this audience, how big is it, and what do they need? Are we talking students during their first differential equation course? I’d say MTK and DynamicalSystems.jl target that quite well along with George’s book. Are we talking people interested in physics but don’t know the equations? What are they modeling and how many people are like that? I’m just really confused as to what this middle group is.
Or are you saying that there aren’t any end users of SciML at all, because the definition of end user in the thread isn’t the standard English one (i.e. “target audience of a good or service”, so the end user of ModelingToolkit being a high-level modeler) but instead means something different like “end users are users of a programming language who are doing it for simple normal tasks”. That’s what I see here:
Which of course isn’t the definition of end user, but this thread would make sense if we used a different definition like this? If the words are just being misused, then a statement like
makes more sense. If someone is a modeler, ModelingToolkit and EasyModelAnalysis are 100% intended to be canned end-user toys and if they aren’t doing that then they are doing something wrong and should be overhauled. But if we’re considering end-users as someone outside of their target audience, then sure I agree with you that they aren’t targeted at that user but end-user implicitly implies target audience of the thing being discussed .
My educational background is in chemistry, and I have been working in polymer physics using numerical tools for more than ten years now. I also published several fast numerical algorithms for solving problems in my field, such as the exponential time difference RK algorithm, Anderson acceleration, and other optimization tools. However, I still find these lines very hard to understand. I don’t think it is at the tutorial level. For example,
What is this? Why should I know about it so I can follow the tutorial? I think a tutorial is for teaching and learning, not marketing. All these lines sound more like academic writing than a user-friendly tutorial.
BTW, I have long heard that SciML is quite powerful and amazing. But each time I try to read the tutorial and doc, I find they are too hard to follow and fail miserably.
At the risk of repeating myself - not everyone that is a potential Julia user is automatically interested in doing physics. There are LOTS of people who aren’t interested in doing physics, or differential equations for that matter, at all - what, if anything, can Julia offer here? What can MTK offer here? Not a whole lot, which is why I keep saying that MTK has good docs, but they aren’t useful at all to someone not interested in doing physics. So your insistence/repeated focus on what MTK and DiffEq can do is a bit irritating - there’s not a whole lot those can do, because not everyone is interested in doing what those tools are for. And that’s fine - but that’s also what this thread is about, to investigate where we’re lacking in terms of non-scientific stuff.
I think we’ve repeated that point quite a lot now - yes, we’re not talking about the target group of MTK/SciML, we’re talking about the target group of Julia at large. As a general purpose programming language, that surely doesn’t limit us to MTK.
I gotta say, it’s quite off putting to see this insistence on doing differential equations/physics/modeling stuff when there is a whole world of things people can & want to do with the language that just isn’t scientific in nature. The differential equations stuff is great - but not everyone is doing differential equations, especially outside the Julia bubble.
A small intervention from someone who wants the language to thrive: I really enjoy the interaction on this thread; this might be translated to a big step in Julia reaching out to all those who are interested in a programming language as a tool for getting things done.
I sometimes feel that many of the Julia selling points (with all its great SciML ecosystem) are not even trying to reach the people who code population.
We have here one of the best general programming languages, and the outsiders are somehow blinded to its greatness exactly by the narrow numerical computing focus. And this might sound like heresy - so know that I don’t want to hurt anybody’s feelings (e.g., I imagine that most of the people around here are somehow connected to numerical computing - myself included).
I think I am starting to drift away now - and this might be the start of a different topic. However, I think is worth asking ourselves what we mean by “next level of popularity”. I don’t think is fair to ask those who are deeply ingrained in the academic domain to speak the language of any programmer who might want to get things done.
So maybe a good question is, how can the gatekeeper (e.g., the mystical creature that is locking people out of the amazing Julia realm) unlock and facilitate the access of this enormous chunk of potential developers?
By the way - if the answer is we are OK with focusing on the numerical computing thing, then things are getting very clear.
However, if we, instead, promote a general programming language, then things are getting harder (and harder doesn’t imply not worth pursuing).
I personally think we are doing a disservice to this great language by not promoting it at its true potential: as one of the best general programming languages available.
On the contrary - I think getting more general purpose programming folk involved with Julia is likely to be a big boon, because all those fancy tools non-scientific (professional and amateur!) programmers are using are surely useful in scientific programming as well. It’s all just programming in the end.
By getting harder, I mean that it might be difficult for a community focused on numerical tools and naturally biased toward a certain jargon to reach outside.
I want to be extremely clear: I am 100% percent for promoting Julia as a general programming language that excels at numerical computing! I’ll edit the other post to reflect that.
The OP is concerned about Julia popularity. I don’t think there’s a problem here. I’ve Googled around a fair bit whilst reading this topic and my conclusion is that Julia is (still?) in the breakthrough phase. Learning a new language requires effort but more and more people are hearing about Julia and asking “Is it worth the effort in switching from my current language?”, and/or “My company has made considerable investment in (say C++ or Python), what is the risk of including another language?”. I think Julia is doing great great for just being in the news and forcing people to consider it. It’s naive to think it’s suddenly going to be number one. And very wrong to think about a shot in the foot like Python endured going from 2.7 to 3.0.
I think this thread is another repeat of this tired old argument.
My research is mainly in deep learning so I do almost all of my programming in pytorch. For years I have been drawing plots for my experiments via matplotlib. However, I really enjoy working in Julia so I try to use it anytime I can.
I’ll probably stick with pytorch for deep learning, but I have been using Makie.jl for last few days and I am seriously considering ditching matplotlib and switch to Makie completely.
Drawing plots is usually quite separated from the rest of one’s work, so I think this might be an opportunity to bring Julia to a wider user base. No matter what one’s primary language is, they can still use Julia and Makie to draw beautiful plots.
One assumption that is being made frequently across this entire forum, not just this thread, is that there is a certain thing (or things) that Julia is missing that would suddenly cause a tidal wave of adoption. I don’t think this is realistic. The most common response I get when I talk about Julia is, “I do X. Can Julia do X?” The problem is that everyone’s X is different. There is no one X that would suddenly shoot Julia past MATLAB, R, Python, Java, C/C++. Julia could release cross-compilable executables tomorrow and folks would still say, “yeah… but I already have code in this other languages that works, so I’m not switching.”
I would also venture to say that a lot of the “Julia would be more popular if…” statements are really a way of saying, “I want to use Julia, but we’re a Python company/university/etc so I’m an outsider.” If that’s the case, then overall popularity doesn’t matter! And the problem is actually simpler than language-level changes! Just write REAL code to solve REAL problems, show your colleagues, and maybe someone will see value in switching. That’s the only way, and it still might not change anything.
In my field there is a huge space for using tools by non-programmers, which end to be an entry point for programming in general. We do molecular dynamics simulations. We don’t develop the packages that perform the simulations, but we do develop analysis tools. The typical student may not develop either, he/she uses the simulation packages and uses the analysis tools.
Having a good package for analyzing simulations is the entry point to start scripting in a language, to organize the analyzes. The user base of such packages is large (for example this one in python, and it is just one of many).
This is the kind of package that is the directly the front-end for users which will not/may not program at all, but are entry points for starting to learn the language.
We have the plan to progressively put packages (and maybe at some point a largeish collection of packages) for the users within our field, and we are working on it (ex1, ex2, ex3, ex4, ex5, etc.).
Some of this has to be training “the next generation” of developers. A lot of the people here are likely open to learning new computer languages, trying new things, etc., but the vast majority of people that I work with are opposed to switching to “new” things because they feel that the time invested in the “old” would be lost somehow - or that the time spent getting good at the new was just simply too long. If your employer is not open to you spending time getting good at Julia on the clock, your only other option is to spend time learning Juila that could otherwise be spent as quality time with loved ones.
Documentation, training, books, conferences, etc. ( … and there are some)
Excellent software ( … and there are some)
University courses that use Julia instead of some other corporate thing with cheap student licenses
Frankly, NOBODY would use Matlab if the license cost came out of their salary.
I saw in my email a post containing “…Julia 2.0…” and I opened it with dreadful interest and a slight sinking feeling. The Python community suffered a lot when switching to 2.0. There are valid reasons for a breaking change; but by definition, a breaking change breaks a lot of code. That’s what came to mind when I read your post’s title.
.