Hi, first of all let me say that I know this is a big subject, but I couldn’t find any serious discussion about it, so I’m asking here. I think it’s obvious that Julia has a really good philosophical basis, and I believe it has a lot of potential to grow and compete with all the other big languages. However, I believe that presently, it’s still not ready to be used for most programming projects. But the “good news” is that perhaps it won’t be so hard to fix, if we make it a priority. I would say there are 3 main roadblocks that I’m aware of: Performance, or: No integration of JIT and interpreter Right now, every function that runs even once, gets compiled. This is causing really long start-up time, and a really long time-to-plot, which could be easily avoided by compiling only functions surpassing some call count. Julia already has an interpreter, so perhaps this won’t be so difficult? I’m sure many would be happy to have that as an option. I should note that the idea has been mentioned here, but I couldn’t find a discussion about it: A Julia interpreter and debugger Another thing that can really help performance, is caching the compiled functions, and luckily there’s already a mechanism to serialize and unserialize them (iirc tim already opened an issue about it). Imports Julia’s import system is probably its weakest point, in terms of user-facing design. Having to literally include files, and then import them separately, is awkward and inefficient… Every modern language (and even most non-modern ones) lets you include your module using a single command, and makes sure to remove duplication for you. The funny thing is that you can import a local module, if you just add the local directory to Julia’s path (which creates its own stack of problems). But perhaps it means that it would be very easy to implement? edit: Just to clarify, I’m talking about importing modules within the same project (local modules), for example a utility module, or a module for common structs. Executable size When compiling a trivial program on windows, the build ends up taking about 500 megs, mostly in complementary dlls and such. Python does the same trick of including itself, but needs only 30 megs. Perhaps there’s a simple solution to drastically reduce the build size? This one is perhaps not so immediate, but all-in-all wouldn’t it be nice if you could send someone a julia program over email, like you can with any other big language? I have a lot of other tidbits that I would like to improve in Julia, but I’ll save it for another time. I’m writing all this not to complain, but to help point out what I think should be on high-priority if we want Julia to make its steps towards mainstream and professional use. I wish I could do all of this myself, but this isn’t one man’s work, even if I was familiar with the code-base and had the time to see it through. But if we decide to make this a priority, we can all carry a little bit and get it done together. So, at this point in time, I’m just trying to figure out if enough people agree with me (or perhaps if I’m wrong about it). So, let’s discuss it. Do you think these issues are important? Do you feel there are other important things missing? I’m interested in your opinion!

I think Julia would be better off maintaining it’s focus on scientific computing (which is already a large and diverse application). There are always trade-offs to be made in language design and shifting Julia’s focus towards general-purpose computing would likely hurt some of it’s strengths when it comes to scientific computing.

If you want to bypass the package manager altogether, you can include the file containing a module declaration and do using .ModuleName But if there’s a mutual dependency, I’ll get WARNING: replacing module Whatever and who knows what kind of issues that might arise due to duplication. It’s just astonishing how unpolished the module system is in Julia compared to other languages. Btw, I’m talking about importing modules within the same project, for example a utility module, or a module for common structs.

How would the goals I outlined hurt any of its strengths? I can’t think of a reason…

[image] erezsh: Having to literally include files, and then import them separately, is awkward and inefficient… Are you sure that you are talking about Julia? Did you by any chance miss the everything about its package manager?

[image] erezsh: But if there’s a mutual dependency, I’ll get WARNING: replacing module Whatever and who knows what kind of issues that might arise due to duplication. It’s just astonishing how unpolished the module system is in Julia compared to other languages. You only get a warning if you are trying to replace an existing module. I don’t understand why you think the module system is unpolished, I find it to work extremely well since even before Julia 1.0. Is there something specific you are trying to achieve that you are unable to do? It sounds to me a bit like you don’t know how to do something, perhaps we can help? [image] robsmith11: There are always trade-offs to be made in language design and shifting Julia’s focus towards general-purpose computing would likely hurt some of it’s strengths when it comes to scientific computing. I agree with @erezsh in that I don’t see why one would need to sacrifice any of Julia’s strengths in regard to scientific computing in order to make it useful in the general case. It seems to me that it’s already most of the way there. It seems to me that the only serious limitations of Julia in regard to “general purpose” computing are Need to constantly re-compile. Binary size (seems like a smaller issue to me, but certainly it can be a huge problem in some applications). as have been pointed out. These are of course issues for scientific computing as well.

[image] erezsh: I couldn’t find a discussion about it [image] Compiler work priorities Internals & Design In response to I thought I might write a little post about the rough priorities of the compiler team: Correctness finding and fixing compiler and inference bugs Multithreading non-copying task stack switching (done) new PARTR parallel runtime locks for I/O operations other thread safety Compile-time latency, aka “the time-to-first-plot problem” making compilation faster caching more things Compiler-related packages and tools PackageCompiler Debugger Cxx type checking/linting J… ideas for reducing compiler latency via the interpreter opened 12:22AM - 20 Sep 19 UTC (UTC) [timholy] timholy latency This is a parallel issue to #33326, focused instead on the interpreter. The basi…c idea is that one of the best ways to reduce inference time is to not infer at all. It's worth noting that [JuliaInterpreter](https://github.com/JuliaDebug/JuliaInterpreter.jl) may be a useful playground for a subset of these ideas. ideas for improving inference and optimizer performance opened 07:09PM - 19 Sep 19 UTC (UTC) closed 04:15PM - 10 Mar 23 UTC (UTC) [JeffBezanson] JeffBezanson inference optimizer latency Many of our latency issues (e.g. time to first plot) primarily consist of infere…nce time. This issue is for documenting and tracking ideas for improving this. I'll try to put ~one idea per comment. I am not sure that it is necessary to start another separate discussion.

[image] erezsh: Performance, or: No integration of JIT and interpreter Even when you build an executable? I haven’t played with building an executable but I had assumed it compiled the code it could at that point. [image] erezsh: Imports As others have mentioned not sure what you mean by this. I guess you could try to compile the “import” and “include” syntax but I usually end up with a “main” file that does all the includes and the individual files just do imports as needed. And if a module is “generic” it gets moved into it’s own source tree/git and I just use the package management provided by Julia. [image] erezsh: Executable size Last I checked the executable size was huge. However I did notice that all the system libraries ended up copied to the build directory even if they were not used. When I looked at the files the executable actually linked to there where only two. But the combination of the executable and the libraries was larger than Go programs which I thought where huge. The executable is still a work in progress so it should get better in the future. I am using Julia as a general purpose language, nothing to huge so I don’t think I’ve gotten hit with any outrageous startup times. However mostly I’ve been using Julia as the server side for web applications so the UI code is all in TypeScript and HTML while the server side just needs to handle the backend data manipulation, thus keeping the Julia code smaller than if everything was in Julia.

[image] erezsh: However, I believe that presently, it’s still not ready to be used for most programming projects. Thanks for your comments, I always learn from these views. But, hard to discuss without clarity around what “most programming projects” are. The dynamic interactivity with developing a ‘program’ in the REPL or a notebook for computational or data science ‘programming’ has been awesome! Plus, “most programming projects” is coupled to concept of “most programmers” Are you considering entire world in your scope? One other point of reference from @jeff.bezanson is this Tube that I found most fascinating, again learning about Julia and the broader ‘computer science’ space: " JuliaCon 2019 | What’s Bad About Julia | Jeff Bezanson" [JuliaCon 2019 | What's Bad About Julia | Jeff Bezanson] It truly feels great to be part of such a humble community, and believe this is a key factor if one were dropping chips on bets for the future. It seems most here don’t see an either / or, but embracing the best from the diversity of the broader ‘programming ecosphere’ (pycall, rcall, cwrappers etc.) But, I’m still incredibly ignorant at this point and so can’t discuss substantively. [image] ExpandingMan: You can point the package manager to a particular directory with ]dev . Oh interdasting! You just might have helped me crack a little nut I was wrestling with last night - thanks!!

[image] pixel27: Imports As others have mentioned not sure what you mean by this. I’ll chime in a little on this. Personally, I found the pacakge manager extremely confusing to use and had quite a bit of trouble understanding how it worked when reading through the manual. In fact, it frustrated me enough to rewrite one of my projects that was functional in Julia into R/C++. It’s very important to note that I came to Julia from R. For recent projects, I’ve had to pick up Python. In doing so, I had to learn all about these things called “virtual environments”. After learning about that, suddenly, a lot of the super confusing things about Julia seemed to make sense. I haven’t dived back in yet, but it’s very possible that having a better grip on virtual environments will make the package manager and imports more intuitive, such as to discourage me from doing stuff like taking one script in a directory, putting it somewhere else and expecting it to run. I haven’t had time to look back into Julia to know if all my confusion would have been resolved yet, but it’s certainly possible. Now, I’m not saying that the manual is definitely missing key information, but it was certainly a very large hurdle for me. I think having a stronger Python background would have made the Package Manager more intuitive.

Could you point out a few risks? I see absolutely none.

Discussion: Plans for Julia as a general-purpose language?

General Usage

Tamas_Papp November 8, 2019, 3:54pm 8

I am not sure that it is necessary to start another separate discussion.

Topic		Replies	Views
Not only for technical computing: changing the narrative around the usecase for Julia Community	52	12632	February 14, 2019
Julia motivation: why weren't Numpy, Scipy, Numba, good enough? Community history	120	85840	September 21, 2018
Ensure Julia is used to its full power New to Julia question , first-steps	100	8602	December 11, 2020
Question about Julia's goals Community question	41	3785	October 27, 2023
What don't you like about Julia for "serious work"? Community	176	27991	April 20, 2024

Discussion: Plans for Julia as a general-purpose language?

Related topics