Memory allocation when evaluating a function inside struct

General Usage Performance
1

I am trying to understand the following memory allocation behaviour:

struct justfunc
  f::Function
end

func = justfunc(sin)
@allocated func.f(0.1) # 32
@allocated sin(0.1) # 0

Having an other variant where the type is inferred

struct justfuncnew{F}
  f::F
end

funcnew = justfuncnew(sin)
@allocated funcnew.f(0.1) # 16 (reduced from 32 , but still there)

I have observed this happening on functions wrapped in struct and this an MWE, but I don’t understand why and where is the memory being allocated? I checked the @code_warntype of the above two calls in the first snippet to check if there is any type-instability, but couldn’t trace out any difference. Insights will be really helpful. Thanks a lot!

2

The first part (32) has allocations because f::Function is an abstractly typed field that needs to point to any subtype of Function, so func.f’s type cannot be inferred. The type has to be checked at runtime a.k.a. dynamic dispatch, which requires allocations as far as I know.

The second part (16) don’t have the aforementioned allocations because f::F is concretely typed and inferrable. The remaining allocations are because funcnew in the global scope is uninferrable. If you did const funcnew = justfuncnew(sin), then funcnew.f(0.1) would have no allocations. Alternatively, you could have wrapped the code in a local scope, like:

let
  funcnew = justfuncnew(sin)
  @allocated funcnew.f(0.1)
end
6 Likes
3

Thank you very much for explaining it clearly Benny :slight_smile: Yes I added const now and resulted in no allocations. So is the let block similar to the function being in a module?

4

I want to say that a let block’s local scope is very different from a module’s global scope and very similar to a function’s local scope, but I’m not sure what you mean by “the function” there. You didn’t define a function.

1 Like
5

Ohk… I see. The question I had was, we might not be able to use let block in our code (or it is very commonly used? I have very little experience with it), but usually have functions and modules, so in practical code I would have to put the struct that was defined above in a module, would it result in the same allocation behaviour then? All in all, I was curious on how would we use the above struct while developing a package such that, when it is instantiated and used for computation, there are no allocation…, is let block the go to solution for this? Thank you for response :slight_smile:

6

I wouldn’t say the let block is a go-to solution for anything, it’s just a quick way to make a local scope that runs without any extra details. A function that is called later is more typical.

I’m just confused what you mean. It’s expected to define structs and functions at global scope, which includes modules. It’s rare to define them in local scope, and some ways aren’t even allowed. Putting things in or out of modules isn’t a method of tuning allocation behavior, and I’m confused why you brought it up because neither of us defined a module. Maybe you’re mixing up some terminology here?

What does affect allocation behavior is if the compiler can know the types of variables. You should start by reading about global variables in the Performance Tips, and that should explain the 2nd part. (By the way, structs and functions in the global scope are assigned to implicitly const variables, so that’s why they don’t ruin performance). But there’s a bit more to variable inference, and I don’t know a single contained source that explains all this.

2 Likes
7

Thank you for your reply. I think I ll read through the Performance tips section and also understand how well-established packages are structured. Yeah may be I mixing up terminology, but thank you for the suggestions and insights into this, was very helpful. Yeah having a good resource for the variable inference mechanism would be instructive to write better code.