This is usually good advice, but it doesn’t always mean you should completely avoid touching the heap. Sometimes it just means you should avoid repeated allocations and instead allocate reusable containers once before entering the hot loop. A corollary is that in any piece of code for which you care about microbenchmarks (i.e., you’re using BenchmarkTools.jl rather than just @time) you should consider taking preallocated memory as an argument, rather than performing heap allocations within that code itself.
I happen to have programmed in the time when the maximum available RAM was < 640 kB, so I know the value of reusing memory 
1 Like
Great! But I think this is a bit different. We’ve usually got plenty of RAM, so it’s not about avoiding OOM, it’s about not wasting CPU cycles on the allocator/GC in performance sensitive parts of the code. Just sharing my interpretation of the frequent “avoid allocations” admonitions, which I don’t think should be understood as “avoid the heap”.
2 Likes
I think it’s the context and how stack/heap allocations are used in practice. Personal computers in the last couple decades can allocate quite a lot of memory, whereever it is, and Julia is one of the languages that needs a lot to just work (well, we’ll see where juliac goes). People in this world tend to worry about heap allocation or GC latency, which stack allocation doesn’t need to deal with. It’s also generally harder to allocate bigger things on the stack because nobody really wants to do that; the heap allows dynamic sizing and easier sharing across methods. Smaller stack allocations in this context are practically almost free; you’d usually need runaway recursion to hit the limit. Even hefty SVectors often don’t crash our systems, it just results in worse performance (compilation, copying).
If more people reach these or smaller systems with juliac, I expect stack allocations aren’t going to be perceived as free as they are today, and maybe we’d get some way to profile them on a bigger system.
1 Like
Off-topic, but my TRS-80 Model III, purchased in 1980, had 4 KB RAM. I programmed it in Z80 assembler and every byte was precious.
7 Likes
Agreed, plus I’d say it’s more specific than that with Julia. The “avoid allocations” should really be read as “avoid (unexpected) allocations due to type-instabilities”. There’s nothing inherently wrong with having code that does lots of heap-based allocations, as long as you can manually manage them in a way that doesn’t impact performance too much (e.g. in C++ by using pre-allocated pools to reuse blocks, instead of calling malloc()/free() every time). But type-instability in Julia introduces allocations outside of direct user control. Hence the attention to type-stability to “avoid allocations”, especially unexpected allocations.
1 Like
Related: the second positional, optional, argument to the Task constructor, determines the stack size for the created Task, when provided:
Might be useful for testing (set a low stack size explicitly) or perhaps preventing stack overflows in tricky recursive code. That said, spawning many tasks with an explicitly chosen size, when the size is not very small, is a bad idea, because, as far as I understand, explicitly choosing a stack size is somehow less efficient, causing the stack memory to get eagerly allocated, in contrast to the default behavior. It’s possible I’m not understanding it correctly, though.
2 Likes
I think the classic ‘avoidable allocation’ issue in Julia is slicing arrays and updating out-of-place when it could be in-place, like for example
for _ in iter
A = A[:, :].^2 + A[:, :] * 2
end
instead of
for _ in iter
A .= A.^2 .+ A .* 2
end
2 Likes
Maybe, but that’s relatively easy to find and fix. Some type-instability issues can be quite subtle and unexpected (at first) and annoying to fix, e.g. iterating rows of a DataFrame (Why DataFrame is not type stable and when it matters | Blog by Bogumił Kamiński).
Sure, but that’s what, in my experience, we are talking about 60% of the time when counseling people to avoid allocations, was my point.
1 Like
Type instability is important, but it’s a separate point from the usual admonitions to avoid allocations, which refer to things like creating new arrays in every iteration of a performance-sensitive loop. This can be completely type stable and still greatly harm performance.
(That said, I agree that the hunt for allocations can help people find and fix type instabilities because dynamic dispatch shows up in benchmarks as a tiny box allocation. But I’d consider that a happy side effect of avoiding allocations, rather than the main reason people advocate it.)
3 Likes