How to know if object memory resides on stack or heap?

General Usage
1

Is it possible to know in Julia whether an object’s memory resides in stack or heap?

2

There are multiple answers to this question depending on what exactly you are asking.

Semantically, mutable objects are required to be on the heap while immutable objects don’t have to.
For object layout/ABI, isbits objects are stored inline while non-isbits objects are stored as heap references.
However, this does not need to hold in the code, i.e. creating an non-isbits object or even a mutable object does not mean there has to be heap allocation. The compiler is free and will in many cases elide the allocation if it can prove that the object does not escape.

3

I meant if there is a function to gather this information dynamically. That is, I have a variable x holding some object, is there a function like isheap(x) that returns true if x refers to something on the heap?

4

Oh, why do mutable objects have to reside on the heap? I thought only dynamically sized collections were on the heap.

5

Well, as I said, that depends on what you are actually asking. Both mutability and isbits are testable so you can just call the respective functions although there’s nothing dynamic about them. Whether an object is allocated on the heap is impossible to test with a function. FWIW, calling that functions itself will certainly be counted as escaped use.

Because they have pointer identity.

1 Like
6

Is there documentation about what counts as “escaped use”?

1 Like
7

Just what I was going to ask … :slight_smile:

8

Anything that the runtime can’t prove to be non-escaping… It’s an optimization detail that’s constantly changing (and I’m currently working on multiple changes that can hopefully make it more consistent). Also note that even if an object doesn’t escape, we will never be able to guarantee that the allocation can be elided, we can only guarantee that if an object does escape than it will be allocated on the heap. OTOH, cases that don’t escape but are allocated can be reported as performance bugs.

Keeping that in mind, what count as escape isn’t too different from other languages, i.e. if the identity/address of the object is used in an unknown way than it’s escaped. This also shows why it’s very hard to give a definition since as the compiler gets smarter the “unknown” set will get smaller. There are cases where we guarantee that the object identity will escape, this includes,

  1. ccall with argument type of Any or Ref of abstract type. (apart from certain runtime c functions known to the runtime)
  2. Store to global.

There are also cases where the pointer and therefore the layout of the object esape but not it’s identity, i.e.

And there are cases that’s currently considered escaping but may not in the future,

  • used in function call to non-inlined function
  • stored to argument of non-inlined function

Additionally, the current optimization pass is also affected by the exact structure of the AST which causes `enumerate` allocates for arrays of mutables · Issue #16190 · JuliaLang/julia · GitHub and that’s the issue I’m currently working on.

3 Likes
9

Right, this is what I was concerned about. This old advice on using non-inlined functions as function barriers will, if I understand correctly, force even immutable structs to be stored on the heap if they are passed through the function barrier.

Okay, good to know.

10

No that’s never the case.

11

Then what is the case? Maybe the contrapositive, that this optimization may not kick in when a mutable struct is passed through a function barrier?

I wish this and other optimization-relevant facts about the compiler were documented somewhere. Some of them are in the “performance tips” section, but some other tips are just folklore.

12

It is currently the case but @noinline is not supposed to affect the behavior of the program so no it’s not guaranteed.

For developers they are usually documented in the comments of optimization that uses them and sometimes in devdoc. For users, these are usually documented in the form of recommanded practice, i.e. performance tip. The two cases I’ve mentioned previously that guarantees escape are worth documenting since that’s a user visible API. I’m not sure how much we want to document the current limitation of the compiler since it is known that people exploide these behaviors incorrectly which can turn a non-breaking change to appear breaking. (replace unsafe pointer operations with their gc-safe Ref equivalents by vtjnash · Pull Request #428 · JuliaIO/HDF5.jl · GitHub)

1 Like
13

Just wanted to say thank you for tackling https://github.com/JuliaLang/julia/issues/16190, https://github.com/JuliaLang/julia/issues/10442 (https://github.com/JuliaLang/julia/pull/22684), and similar issues! Having to work around these issues in high performance code has been a bit painful, so I’m very excited about the upcoming changes made by yourself and others.

14

FYI none of the existing PR’s fixes `enumerate` allocates for arrays of mutables · Issue #16190 · JuliaLang/julia · GitHub, that’s my current test case for up coming ones though.

1 Like