# PythonCall allocations

**URL:** <https://discourse.julialang.org/t/pythoncall-allocations/128886>\
**Category:** Profiling\
**Tags:** question\
**Created:** [May 9, 2025, 11:29pm UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886 "2025-05-09T23:29:16Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Refaat\_Ismail](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/refaat_ismail/32/215929_2.png) [@Refaat\_Ismail](https://discourse.julialang.org/u/Refaat_Ismail)\
**Post date:** [May 9, 2025, 11:29pm UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/1 "2025-05-09T23:29:16Z")

</div>

I am trying to understand how to minimize allocations in PythonCall. Here, I am using stim, a python library, to generate samples (stored in numpy arrays). I have two implementations:

```julia
function generate_samples(;circ_string, num_samples=1, batch_seeds_ranges=[1:100, 150:200] )
    PythonCall.@pyexec (circ_string, num_samples, batch_seeds_ranges) => """
    import numpy as np
    import stim
    # Set up the circuit and sampler
    circ_stim = stim.Circuit(circ_string)
    nshots= sum(len(seeds) for seeds in batch_seeds_ranges) * num_samples
    ndets = circ_stim.num_detectors
    nobs = circ_stim.num_observables
    det_shots_np = np.zeros(shape=(nshots, (ndets + 7) // 8 ),dtype=np.uint8)
    obs_shots_np = np.zeros(shape=(nshots, (nobs + 7) // 8),dtype=np.uint8)
    counter = 0
    for seeds in batch_seeds_ranges:
        for idx,seed in enumerate(seeds):
            sampler = circ_stim.compile_detector_sampler(seed=seed)
            sampler.sample(num_samples, separate_observables = True, bit_packed=True, dets_out=det_shots_np[counter:counter+num_samples], obs_out=obs_shots_np[counter:counter+num_samples])
            counter += num_samples
    """ => (det_shots_np, obs_shots_np, ndets, nobs);
    return det_shots_np, obs_shots_np, ndets, nobs
end

function generate_samples_wrapper(circ_string, num_samples = 1, batch_seeds_ranges=[1:100, 150:200])
    circ_stim = stim.Circuit(circ_string)
    ndets = circ_stim.num_detectors
    nobs = circ_stim.num_observables
    nshots= sum(length.(batch_seeds_ranges)) * num_samples
    det_shots_np = np.zeros(shape=(nshots, Int(ceil(pyconvert(Int, ndets)/ 8)) ),dtype=np.uint8)
    obs_shots_np = np.zeros(shape=(nshots, Int(ceil(pyconvert(Int, nobs)/ 8))),dtype=np.uint8)
    counter = 0
    for seeds in batch_seeds_ranges
        for (idx,seed) in enumerate(seeds)
            sampler = circ_stim.compile_detector_sampler(seed=seed)
            sampler.sample(num_samples, separate_observables = true, bit_packed=true, dets_out= det_shots_np[counter: counter+num_samples-1], obs_out= obs_shots_np[counter:counter+num_samples-1])
            PythonCall.Core.pydel!(sampler)
            counter += num_samples
        end
    end
    return det_shots_np, obs_shots_np, ndets, nobs
end

julia> @time det_shots_np, obs_shots_np, _, nobs= generate_samples(;circ_string);
  0.130245 seconds (156 allocations: 3.430 KiB)

julia> @time det_shots_np, obs_shots_np, _, nobs= generate_samples_wrapper(circ_string);
  0.140203 seconds (8.38 k allocations: 178.477 KiB)

```

I do not understand why the second implementation has such a higher allocations count. Any insight would be appreciated.  
Using the first implementation would be nice, but unfortunately I run into errors when I try to run it on multiple workers.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [May 9, 2025, 11:44pm UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/2 "2025-05-09T23:44:32Z")

</div>

Put your code in triple backtick blocks, like this  
```  
```  
Not only would it be formatted to be readable, you also wouldn’t accidentally tag other users with `@`. Incidentally, isolated expressions like that are put in single backtick quotes like ` `, and you can read them because I’m escaping special characters with \.

From what I can eyeball at the moment, you’re getting overhead from the numerous `PythonCall` wrappers crossing the language barrier; by comparison, the `@pyexec` code crosses from Julia to Python back to Julia. There may also be extra overhead from some wrappers not being type-stable across the language barrier, which you can check with `@code_warntype`. Also note that Julia is unable to profile allocations that occur in Python itself, so depending on how large the Python allocations are, this extra overhead may not even matter.

---

<div class="post-metadata">

**Author:** ![Refaat\_Ismail](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/refaat_ismail/32/215929_2.png) [@Refaat\_Ismail](https://discourse.julialang.org/u/Refaat_Ismail)\
**Post date:** [May 10, 2025, 12:48am UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/3 "2025-05-10T00:48:01Z")

</div>

Thanks for the formatting advice! very helpful!  
This is what I get when I profile allocations for the julia wrapper one:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/a/fa41ded8cfb1a9b9ef5870af8d5afa4e53dc0926.png)

If I understand correctly, the both functions might be doing similar number of heap allocations. But the allocations made by Python are not counted, that’s why mostly Python-based function has lower count?

I also tried to use `\@code_warntype` and it does not show an issue for now:

```julia
julia> @code_warntype generate_samples(;circ_string)
MethodInstance for Core.kwcall(::@NamedTuple{circ_string::String}, ::typeof(generate_samples))
  from kwcall(::NamedTuple, ::typeof(generate_samples)) @ Main /pscratch/sd/r/rismail/mwe_stor/mwe_soft/scripts/play.jl:34
Arguments
  _::Core.Const(Core.kwcall)
  @_2::@NamedTuple{circ_string::String}
  @_3::Core.Const(Main.generate_samples)
Locals
  @_4::Union{Int64, String, Vector{UnitRange{Int64}}}
  circ_string::String
  num_samples::Int64
  batch_seeds_ranges::Vector{UnitRange{Int64}}
Body::NTuple{4, Py}
1 ── Core.NewvarNode(:(@_4))
│ %2 = Core.isdefined(@_2, :circ_string)::Core.Const(true)
└─── goto #3 if not %2
2 ── (@_4 = Core.getfield(@_2, :circ_string))
└─── goto #4
3 ── Core.Const(:(Core.UndefKeywordError(:circ_string)))
└─── Core.Const(:(@_4 = Core.throw(%6)))
4 ┄─ %8 = @_4::String
│ (circ_string = %8)
│ %10 = Core.isdefined(@_2, :num_samples)::Core.Const(false)
└─── goto #6 if not %10
5 ── Core.Const(:(@_4 = Core.getfield(@_2, :num_samples)))
└─── Core.Const(:(goto %15))
6 ┄─ (@_4 = 1)
│ %15 = @_4::Core.Const(1)
│ (num_samples = %15)
│ %17 = Core.isdefined(@_2, :batch_seeds_ranges)::Core.Const(false)
└─── goto #8 if not %17
7 ── Core.Const(:(@_4 = Core.getfield(@_2, :batch_seeds_ranges)))
└─── Core.Const(:(goto %24))
8 ┄─ %21 = (1:100)::Core.Const(1:100)
│ %22 = (150:200)::Core.Const(150:200)
│ (@_4 = Base.vect(%21, %22))
│ %24 = @_4::Vector{UnitRange{Int64}}
│ (batch_seeds_ranges = %24)
│ %26 = Base.keys(@_2)::Core.Const((:circ_string,))
│ %27 = (:circ_string, :num_samples, :batch_seeds_ranges)::Core.Const((:circ_string, :num_samples, :batch_seeds_ranges))
│ %28 = Base.diff_names(%26, %27)::Core.Const(())
│ %29 = Base.isempty(%28)::Core.Const(true)
└─── goto #10 if not %29
9 ── goto #11
10 ─ Core.Const(:(Base.kwerr(@_2, @_3)))
11 ┄ %33 = Main.:(var"#generate_samples#23")::Core.Const(Main.var"#generate_samples#23")
│ %34 = circ_string::String
│ %35 = num_samples::Core.Const(1)
│ %36 = batch_seeds_ranges::Vector{UnitRange{Int64}}
│ %37 = (%33)(%34, %35, %36, @_3)::NTuple{4, Py}
└─── return %37

julia> @code_warntype generate_samples_wrapper(circ_string)
MethodInstance for generate_samples_wrapper(::String)
  from generate_samples_wrapper(circ_string) @ Main REPL[5]:1
Arguments
  #self#::Core.Const(Main.generate_samples_wrapper)
  circ_string::String
Body::NTuple{4, Py}
1 ─ %1 = (1:100)::Core.Const(1:100)
│ %2 = (150:200)::Core.Const(150:200)
│ %3 = Base.vect(%1, %2)::Vector{UnitRange{Int64}}
│ %4 = (#self#)(circ_string, 1, %3)::NTuple{4, Py}
└── return %4

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 10, 2025, 12:58am UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/4 "2025-05-10T00:58:14Z")

</div>

> [@Refaat\_Ismail](#):
>
> But the allocations made by Python are not counted, that’s why mostly Python-based function has lower count?

Right.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [May 10, 2025, 6:02am UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/5 "2025-05-10T06:02:05Z")

</div>

Your `@code_warntype` printouts actually don’t show the bulk of what the methods do, just handling `generate_samples`’s keyword arguments or `generate_samples_wrapper`’s default positional arguments before calling the methods that do the real work:

> [@Refaat\_Ismail](#):
>
> ```julia
> │ %37 = (%33)(%34, %35, %36, @_3)::NTuple{4, Py}
> └─── return %37
> 
> │ %4 = (#self#)(circ_string, 1, %3)::NTuple{4, Py}
> └── return %4
> 
> ```

`@code_warntype` also generally does not recursively infer the bodies of other methods being called. That doesn’t matter for `generate_samples` because it’s not going to see anything going on in the Python code string anyway (which is a bit of a relief because tracking down the underlying method of a keyworded method is a pain manually, better use Cthulhu.jl), but you’ll want to check `@code_warntype generate_samples_wrapper(circ_string, 1, [1:100, 150:200])`. Also note that the `Py` wrapper may be technically type stable and look fine, but that’s deceptive; it just affirms that PythonCall is not distinguishing the types on the Python side for the Julia side. Not entirely sure what all of `det_shots_np, obs_shots_np, ndets, nobs` can be, but the first two could be `Matrix{UInt8}` or whatever the `PyArray` wrapper is.

---

<div class="post-metadata">

**Author:** ![cjdoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cjdoris/32/213133_2.png) [@cjdoris](https://discourse.julialang.org/u/cjdoris)\
**Post date:** [May 10, 2025, 9:54am UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/6 "2025-05-10T09:54:56Z")

</div>

If I were you I’d not worry much about the allocations - the two snippets of code run just as fast so why care?

Reducing the allocations is usually only relevant when micro-optimising tight loops where any allocations become a bottleneck, whereas in this case the allocations add a pretty small overhead (≈10% maybe).

---

<div class="post-metadata">

**Author:** ![Refaat\_Ismail](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/refaat_ismail/32/215929_2.png) [@Refaat\_Ismail](https://discourse.julialang.org/u/Refaat_Ismail)\
**Post date:** [May 20, 2025, 3:54am UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/7 "2025-05-20T03:54:06Z")

</div>

My main motivation is that the second implementation using Julia wrappers produces memory leaks that lead to out-of-memory (OOM) events. I’m not entirely sure why this occurs, but I suspect it’s due to allocations that are managed on both the Julia and Python (and C as well since stim uses C) sides without being properly cleared. This is why I was experimenting with reducing such allocations using the pyexec code, and it managed to avoid the memory leak.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [May 20, 2025, 6:47am UTC](https://discourse.julialang.org/t/pythoncall-allocations/128886/8 "2025-05-20T06:47:45Z")

</div>

> [@Refaat\_Ismail](#):
>
> out-of-memory (OOM) events

On the Julia, Python, or C side?