Kwargs... and allocations

General Usage Performance
1

I recently was trying to overload some real-time code (meaning I don’t want any GC or allocations).
I needed to add in some kwargs to the help with overloading some intermediate functions.

This causes allocations because of the named tuples. I ended up resorting to a custom settings structure to handle the job of the kwargs. Didn’t know if there was a better way to do this kind of thing when wanting the flexibility of the defaults of the kwargs but not the allocations and GC.

2

Just avoid keyword arguments if you need allocation free code. While this is a small inconvenience, best embrace it. I always pass a settings struct to the constructor of most of my modeling and control components. Sometimes even two:

@with_kw mutable struct Winch @deftype Float64
    wcs::WCSettings
    set::Settings
    wm::AsyncMachine = AsyncMachine(set)
    v_set     = 0 # input
    force     = 0 # input
    acc       = 0 # output
    speed     = 0 # output; reel-out speed; only state of this model
end
2 Likes
3

Can you give an example that leads to these allocations?

1 Like
4

What do you mean, allocations from kwargs? That’s news to me.

3 Likes
5

Not currently as the example I have is quite complex. I haven’t whittled it down to a MWE. I find debugging allocations to be incredibly tedious in general in julia.

Right now for example, I have a script that I can include which sets up a module with the exact same code I am running in an @testset runtests.jl section. I have removed the kwargs and am using the settings function as suggested by ufechner7 and also ChatGPT earlier today (which is what I have recoded.)

It does not allocate when I run my tester function in the standalone module. The same code inside @testset allocates. There are some const ref pointers I use to share memory to avoid allocations. ChatGPT thinks those could be an issue. But I’ve followed its suggestions. I’m in version 1.11.5 julia.

If anyone has any guesses on where I can look without a MWE, thank you. If it goes on for much longer, I’ll try to figure out a MWE.

To check where things are going, I have to go into the profiler and hope I can understand through pprof what might be happening. Without the code failing the same way… it is very challenging.

Anything the julia community to help in debugging these allocation free code sections, would be great!

6

apparently the splatting and slurping into the Named Tuples can cause allocations. At least that is what ChatGPT claims.

7

I wouldn’t necessarily trust that. I mean, I’m sure it can happen, but it should be avoidable.

2 Likes
8

I’m wondering if I’ve introduced some sort of type instability. Checking that now.

9

Are you splatting and slurping back and forth between namedtuples and kwargs yourself, or is the claim that the implementation of kwargs involves problematic splatting and slurping under the hood?

10

I’m doing the splatting and slurping on overloaded functions so I can just pass the kwargs down to the next level. These intermediate functions are just setting up the underlying function.

11

Can you maybe describe what data types are in the kwargs? IIRC it’s possible that if you are passing Functions or DataTypes that maybe there could be issues.

Even just some pseudocode that also explains your issue further could be useful while you work on an MWE.

12

I see. It’s easy to end up with type instabilities in that kind of code. Would have to see an example to propose a fix/workaround.

13

One unusual type I pass in is an abstract type that is basically there to force a dispatch choice on the underlying function.

abstract type AbstractInterpolationMode end

abstract type InterpMode1 <: AbstractInterpolationMode end
abstract type InterpMode2 <: AbstractInterpolationMode end

This was one of the Kwargs that was passed to the top level function, and it defaulted to one of those.

Then there was a function call that looked something like this:

interpolationHelper!(::Type{InterpMode1}, o, t, p, f; kwargs...) = interpolateMode1!(o, t, p, f; kwargs...)

interpolationHelper!(::Type{InterpMode2}, o, t, p, f; kwargs...) = interpolateMode2!(o, t, p, f; kwargs...)

Some of the kwargs would be mutable structures containing mutable vectors, etc. that could be reused in calculations. If the kwarg wasn’t given, then the memory would be allocated instead, but it could be passed in to avoid garbage collection.

Does that make sense.

I’ve moved all of those off into a settings structure, still battling with allocation checks in the test functions. I think it is an interaction with the @test macro though and these “ref” memory pointers.

I’m attempting to build a dummy module with the ref pointers and the testing functions so the “@test” macro just calls that module and checks for the allocations. When I did this by hand it worked.

There is another top level function that had the InterpType where InterpType<:AbstractInterpolationMode as a kwargs…

That function interpolationHelper! with it passed in as the first argument. Then that InterpType isn’t used in the call to interpolationHelper!

14

Aha, you generally want to pass values, not types. The specialization heuristics won’t specialize on types unless you explicitly capture a parametric ::Type{T}.

2 Likes
15

The top function looks like this.

function dummyTop(obj::mytype{T,N}, var2::xyzVecT{T}, var2::anotherthing{T}, var3::T;
            useNearest::Bool=false,
            interpolationMode::Type{IntType}=DEFAULTINTERPOLATION) where {T<:Number, N<:Number, IntType<:AbstractInterpolationMode}

The T is a Float64 and the N is an Int64.

16

My attempt at backwards compatibility is becoming a nightmare. :expressionless:

17

The reusable version dummyTop! has another kwarg…

reusableMemory::InterpolationMemory{T}=InterpolationMemory(obj)

18

Ok. Since my settings Structure had just as much trouble passing an @allocated check in my @testset as the kwargs until I pulled it off into its own module, I’m wondering if the kwargs version would have passed if I had pulled it out into its own module as well.

I’ll test that after lunch and comment back here if I was testing an interaction between const var=Ref{…} problems in my testing modules.

I have the settings version passing though.

In general, anything we can make debugging where allocations happen easier would be great!

19

Are you aware of AllocCheck.jl?

1 Like
20

I think @mbauman called it: Make sure the interpolationMode kwarg has the parametrized ::Type{IntType} annotation in every method in the call chain. That means you can’t use splatting and slurping to pass this argument through to an inner call. Instead of

function outer(x; kwargs...)
    y = 2x
    return inner(y; kwargs...)
end

you need

function outer(x; interpolationMode::Type{IT}, otherkwargs...) where {IT}
    y = 2x
    return inner(y; interpolationMode, otherkwargs...)
end

The same applies to any other kwarg that takes a type.

3 Likes