Maybe if there are lots of people paid by corporations to drive the language forward, not if the language relies on attracting volunteers.
1 Like
Just want to show you the other side of the coin, so you can get a grasp of what new users sometimes feel. I wrote this post yesterday.
After two days of struggling with that, someone defending Julia or providing an answer that says nothing ends up irritating you.
I also think that sometimes putting a title that says âHUGE BUGâ is kinda strategic, so that people try to explain in depth whatâs going on. Otherwise, the post becomes unnoticed.
3 Likes
I think I can guess which answers youâre referring to. I donât think that person was trying to defend Julia, I think they were very confused and didnât communicate that well. They mentioned a bug (with a discourse thread that I had actually commented on), but deleted the comment when they realized that bug was entirely unrelated to the post. The argument on what counts as a bug wasnât helpful at all because it didnât make clear the aforementioned bug was unrelated until very late, nor did it actually explain the original issue. My suggestion for discourse etiquette there is: 1) if I were to ever contradict myself, I would write strikethroughs instead of deleting the text, 2) always link the section where something is documented, if possible, or name a search phrase specific enough to jump to that section.
You refactored to best workarounds in cases 3 and 4, so you actually got half the answer on your own. I think another commenter was onto something, so Iâll comment on that post later to try to reach the finish line. No promises though, this is an unintuitive issue and Iâm not very smart.
3 Likes
That poster was clearly trying to be helpful, while also having a different opinion than you. If simple disagreement or honest mistakes are going to be out of bounds for posting at this board, I think that would not create a very healthy atmosphere.
And now one can expect ones posts to be held up as examples of negative behavior in other contexts? I donât like this at all.
11 Likes
So basically, âwe should be more understanding of Julianâs flawsâ, at least some. But I donât think the implication was this should be prohibited, just that there could be improvements. We have a post for how to ask questions (e.g. no screenshots, MWE if possible), maybe we should have a post for how to communicate answers (how to reference sources, avoiding links that may break, descriptive text, strikethroughs avoid confusion better than deletion).
3 Likes
if doing things the Julian way still leaves much to be desired [then in our âŠ] Time to first plot is unavoidably long and writing code that âruns like Câ can sometimes be frustrating because it requires almost as much knowledge (sometimes in the form of cool tricks) as writing C code, but you donât actually have full control over memory.
Yes, TTFP is slow by default but avoidable completely, I believe (e.g. with a sysimage, already, even better with tool below), and true about ârequires almost as much knowledgeâ (of low-level machine details, but only for fastest code, hard for any language, not to beat C). In short I think you have all the âcontrol over memoryâ you need, e.g. Libc.Malloc (and disabling GC), while default use good enough. Only Alloca is missing?
What I would like when I make a method/function, is a warning:
That code is not as optimal, as it could be, because of type-instability, memory allocation etc.
It might require you to use the REPL (or some tool for, first, as a prof-of-concept, I would thought want the the default REPL), but would ne needed e.g. in Visual Studio Code, not just for REPL. [Some code might be type stable if called with e.g. Float64, but type-unstable for non-default, say Float32, so this isnât trivial, or clear how complainy the REPL should be, but it should be possible.]
@mikeâs package(s) for sysimages seems excellent, and Iâm still waiting for an announcement, not sure why delayed or not done:
Currently the following system image bundles are provided:
AlgebraOfGraphics, contains the Algebra of Graphics plotting package along with the Cairo-basedMakiebackend, andDataFrames.DataFrames, contains theDataFramesandDataFramesMetapackages.JuliaFormatter, containsJuliaFormatter, with latency reduced enough that users may consider using it as the basis for a command line formatting script for Julia source code.JuliaSyntax, containsJuliaSyntaxand uses the provided precompile scripts to reduce latency and enableJuliaSyntaxas the default parser for all code.Requests for additional images can be made to
curated-system-imagesrepo where the manifests and build scripts are located for the above images.
Almost all the tech flaws (not doc flaws), can be fixed (or already have been, e.g. 9 KB Julia binaries, still with limitations though). Itâs just a matter of knowing the non-default ways.
1 Like
I didnât know about this, this is great. Iâd wanted to initiate an effort towards exactly this, so itâs great to see it already in motion.
1 Like
I donât yet consider it stable, hence why thereâs no official announcement yet. If people like living on the edge then theyâre welcome to give it a go and report issues if they want to 
4 Likes
this is actually a quite good example of ânot understanding Juliaâs flawsâ - itâs not the programmer who is allocating like crazy, itâs the compiler. So in a certain way as julia developer you need some understanding how code is translated into execution and (something like) local assignments create local memory impact. This is clearly a flaw of the julia compiler.
4 Likes
Is it? In a language like C (or Zig), heap allocations are never automatic, so on the flip-side you need to be explicit and have malloc all over the place. Some say itâs a feature of those languages, others would say Julia (and Go) have clarify and less clutter. Julia is this way by design, and Iâm not sure what else the compiler could do. Well it can stack allocate more (and thatâs already done in 100% or 99% of cases possible, but wasnât historically).
If you mean the compiler should act as a linter, I think that might be valuable, while a misplaced criticism of a compiler. For other languages those are separate tool (while often you might have some warnings). I could see the the compiler/Julia/linter-in-REPL warning about allocations, at least for ! functions (the bang means modify in-place, so allocations less appropriate, or never?).
Itâs perfectly valid for other functions to sometimes allocate, not always a concern, more though if in a loop. Some language e.g. D, have @nogc annotaion, to disallow allocation and thus GC (but also disallow calling other non-@gc-annotated functions). Then it IS a defined job of the compiler, but only for such annotated functions.
3 Likes
The compiler could be better at escape analysis and stack allocate more. Actually this is being pretty much worked on, so it is not only something where Julia could be better, but probably will.
(a similar thing, of the past, was the allocations caused by views. Julia just got better, good for us).
6 Likes
Holding a similar view to @Palli I would firmly call Juliaâs allocation/freeing mechanisms a feature of the language. Of course, almost every new Julia user will at some point run into unexpectedly slow code as a result of this feature, but perhaps this signals that Julia would strongly benefit from heavily advertised, accessible (purposefully overexplained) content specific to these sorts of issues.
While content like this exists, itâs clear that (most) people donât look for it because they donât expect it to be a feature, so we might want to have this flagged/warned somehow in the REPL/compiler itself. Additionally, content like this exists, but if we want young people[1] to learn the language (which we should imo) then we canât expect them to know how to dig through documentation or watch overly formal videos to find it.
On compiler flags/warnings, I feel like this is where a commercial IDE from a company like JetBrains could be so beneficial. Imagine PyCharm for Julia, call it JuliaSpell[2] or something. A heavily integrated linter/static analyser like JET.jl could potentially be what Iâm after, it would just need a lot of work to make it as smooth and easy to use as possible.
3 Likes
I would propose that, rather than the compiler, the runtime could give a warning if there are « too many » allocations at a moment during the run.
This could be detected by the GC as « abnormal high activity », and possibly reported at end of the run ( rather than at the moment it is detected)(easy to implement with atexit. Advantage is that it doesnât clutter the run. For the REPL, could be reported when coming back to REPL prompt?
It could also be enabled/disable by functions or macros, or glabal state flag. Could also be prototyped as a package, if there is a GC callback? (I donât know of any at the moment 
)
HTH,
3 Likes
New in 1.8:
The new
Profile.Allocsmodule allows memory allocations to be profiled. The stack trace, type, and size of each allocation is recorded, and asample_rateargument allows a tunable amount of allocations to be skipped, reducing performance overhead (#42768)
Itâs not on by default, nor should it be (because profiling has an overhead).
Note, also new (likely only CPU profiling, not memory profiling, but could allow for latter too?):
A fixed duration cpu profile can now be triggered by the user during running tasks without
Profilebeing loaded first and the report will show during execution. On MacOS & FreeBSD pressctrl-tor raise aSIGINFO. For other platforms raise aSIGUSR1i.e.% kill -USR1 $julia_pid. Not currently available on windows (#43179).
and not new:
can be enabled with
GC.enable_logging(true), which causes Julia to log to stderr every time a garbage collection happens.
Allocations can be a performance flaw, but not all are, and itâs hard to make a program without heap allocations. I think it would be crazy talk for say to expect C program to emit warnings for malloc, even if many in rapid succession, because they are normal in a program.
I realize youâre proposing only logging for âhigh activityâ, and that changes the picture a bit, but then ou need to keep track of time, wen was the last one, or in space, if in a loop, and I can see it as a non-default option, not on my default, though maybe allowed to trigger collecting data by default with CTRL-T?
easy to implement with
atexit
The reporting at that time isnât the hard part, if/since you have the structure, but filling it at runtime will slow down, more than emitting to a log as we go along?
2 Likes
I agree that it is complicated to define « high activity ». So, may be just report highest allocation activity (and source line location) at exit?
Highest activity could be e.g. a smoothed value of « number of allocations per second » (on a 1 sec or 5 sec rolling window?
As you said, may be it is too complicated? And we already have (manual) procedure in 1.8. My initial idea was to help the newbie in some simple way, but to have an idea of reasonable threshold warning values, we should have a passive instrumentation ⊠which has no infrastructure and whose result could vary a lot.
You would still need to keep track of all the other lines that allocate, so that you can rank later to show the one with highest.
i might do the wrong example here and pseudocode, because i have not written julia code in years and didnât really run into a âyou allocate too muchâ problem.
letâs loop over a long list of indexes, calculate something, and copy the results into an available array
for k = 1:1e6
value = function(constant parameters, k, some data)
% do something else
% do something else
result_array[k] = value
end
isnât this allocating a bunch of values? If yes, the compiler should reduce to result[k] = function(etc.)
And for heap/stack allocations: Sorry, if i want to differentiate this, i go to a low level programming language (aka C - a form of macro assembler). In a high level language with memory management (aka GC) i donât even want to think about managing this.
1 Like
The JuliaCon22 keynote by Jeremy Howard was an eye-opener for me. Iâve been trying to evangelize Julia to my engineering colleagues, and only a few younger folks seem open to it. I think if we can solve the issues so that âJulia could have Goâs thingsâ as Jeremy puts it here - JuliaCouldHaveGosThings - the language would take off in popularity. Hereâs a screenshot that hammers home a major point:
The inadequate answer offered in the lower-left bubble is admittedly from 2019, but unless Iâve missed some news, the status quo currently isnât a lot better.
If we donât solve these issues, I think Julia will be relegated to a niche language that few of us can use in our jobs. I reluctantly work mostly in Matlab, Python, and shell-scripting (bash / csh) in my day-to-day work. I canât easily give someone a Julia program to run, and itâs not trivial to get a working environment set up.
4 Likes
PackageCompiler.jl is I believe still the go to package to compile, but no longer the only one. Yes, the binaries arenât small with it (but can be tuned smaller), but with other compiler package 9KB small binaries is possible (then you have some limitations).
PackageCompiler was completely rewritten in 2020, and then StaticCompiler was completely rewritten in 2022. To say itâs not much a lot better is just plain incorrect. These days PackageCompiler works quite well with just one line of code and is actually documented (the version way back then was⊠bad). StaticCompiler is still limited in application but is quite good for what it handles now. Are they perfect? No, but they have improved at least 10-fold and are used in real industrial applications all of the time (they are used in all applications of JuliaSim for example, which could not have been possible with the old versions).
For details on PackageCompilerâs transformation, see
13 Likes