Julia allocates when passing variable to function instead of constant

New to Julia
1

Hello everyone, I am very much new to Julia trying to understand how allocations work and how to reduce them. I am writing a small program that reads some files, processes the data and outputs the results. Hence I am reading files line by line and I came across a curious behavior that I cannot explain myself and struggle to find answers to online.

I have a function that takes an index i, a (static) vector sys_size that has some needed information and another vector pos that is set using the other arguments. I have tested this in the REPL that this runs without any allocations (the vectors are of length 2 or 3).

function set_pos!(i, pos, sys_size)
    D = length(sys_size)
    pos[D] = i
    for i in D:-1:2
        x, y = fldmod1(pos[i], sys_size[i])
        pos[i] = y
        pos[i-1] = x
    end
end

Now I want to use this in my main loop over the file that is itself in a function and looks as

for (line_i, line) in enumerate(eachline(joinpath(dir_path, fname)))
    set_pos!(line_i, pos, sys_size)
end

which has allocations of ~2 times the number of lines in the file. When testing this I have ran the same loop without a body (empty) which results in half the allocations and from further testing via the equivalent loop at the end of the post I conclude there is ~1 allocation per iteration coming just from building a String, this makes sense, that means there seems to be an additional 1 allocation per iteration coming from the set_pos! function and that I do not understand.

To further test this I have tried running the loop

for (line_i, line) in enumerate(eachline(joinpath(dir_path, fname)))
    set_pos!(1, pos, sys_size)
end

instead, simply replacing the integer variable by a constant. This somehow reduces the allocations to ~1 per iteration, same as if it had an empty body. This very much confuses me as line_i is just an integer and all I change is which integer is passed to the function. I would greatly appreciate any help with understanding this.

Alternative, more bare-bones but equivalent loop that I also used for testing (it reproduces the same results):

file = open(joinpath(dir_path, fname))
line_i = 1
while !eof(file)
    line = readline(file)
    set_pos!(line_i, pos, sys_size)
    line_i += 1
end
2

What happens if you put that top level code inside a function?

3

If you use global variables in your test loop, then there is a dynamic dispatch on every function call. That’s why you have to measure performance in a function or use a benchmark package that allows you to “interpolate” globals into constants during benchmarking.

2 Likes
4

Sorry, I think wasn’t clear enough but all of this is in a function, I just didn’t want to post the whole thing as it is relatively long. I have a file containing module with two functions - the mentioned set_pos! and analyze_run which contains the said loop. This file is a part of a super simple package, my workflow is to start julia with the package’s environment, use Revise and then import the module via using. Finally, I test any of the functions in the REPL via @time macro. I have also used the package Infiltrator to spawn a REPL when actually inside the loop and even then when I run set_pos!(line_i, pos, sys_size) I get 0 allocations, this is confusing me even further.

When it comes to global variables I do actually have a couple defined in the mentioned module. They are all defined with “const” (which I am actually somewhat confused about too, from what I read online this is more a hint to the compiler about the variables’ type than actually a constant value as in C) and the only ones that come into play with this part of the code are

const it = Int64
const D = 3

these are used in the definitions of pos and sys_size

pos = Vector{it}(undef, D)
sys_size = parse.(it, SA[sys["size_x"], sys["size_y"], sys["size_z"]])

so currently sys_size is always an SVector of length 3 and pos a Vector of length 3 as I do not change D (I will later, this is some temporary stuff).

Thank you for your quick reply and sorry it

5

Try to minimize the reproducer so you could share it.

1 Like
6

Sure, I put together a minimal example with the second loop style for simpler debugging, this should work on any text file with at least a couple of lines

module EigenAnalysis
export analyze_run

using StaticArrays

const it = Int64

const D = 3

function analyze_run(fname)
    # A couple of semi-hardcoded bits for this demo
    if D != 3
        println("Currently only 3D is supported")
        return
    end
    sys_size = parse.(it, SA["100", "90", "110"])

    # prealocate pos
    pos = Vector{it}(undef, D)

    file = open(fname)
    line_i = 1
    while !eof(file)
        line = readline(file)
        set_pos!(line_i, pos, sys_size)
        line_i += 1
    end
end

function set_pos!(i, pos, sys_size)
    D = length(sys_size)
    pos[D] = i
    for i in D:-1:2
        x, y = fldmod1(pos[i], sys_size[i])
        pos[i] = y
        pos[i-1] = x
    end
end

end
7

readline allocates a new string for each line. Similarly for the eachline iterator.

If you want to iterate over the lines while re-using a single buffer for each line you could use ViewReader.jl. (In Julia 1.11 there will be a new copyuntil function that you can also use to read lines into a re-usable buffer, which you can treat as a string with StringViews.jl, which also has optimized support in BufferedStreams.jl.)

4 Likes
8

Yes, I am aware of that, though this made me realize I didn’t actually check the issue works for the minimal working example I posted, there I do seem to be getting one allocation per file line/loop iteration. However with the code I am actually using the allocations halve when I comment out the line using set_pos!.

It seems I just have to share the actual code, I did not find a convenient way to do this within the discourse post so here is a link to download it My code. This includes a small project and a sample input file. To run it just extract the zip and then run “julia --project=EigenAnalysis_small EigenAnalysis_small/main.jl”.

In this example I have duplicated the function analyze_run and in the second copy I have commented the line using set_pos!. For some reason that I am struggling to understand the version that does call it has twice as many allocations, this is what I am struggling with and I am even more surprised that the minimal example I posted before does not behave in the same way.

9

If you want people to help you, it’s best to try to boil things down to a minimal example that reproduces your problem. This is a good skill to practice in any case. (Just keep deleting code until the issue goes away.)

2 Likes
10

Okay, I think I finally got somewhere though I still don’t know what’s going on so I am posting again. I have narrowed it down and it seems the problem only happens once I actually read the sys_size static array from the setup files I am using. Here I have another minimal example, this one actually works though, but it relies on the IniFile package, the julia code is

using StaticArrays
using IniFile

const it = Int64

const D = 3

function analyze_run_noini(s_fname, d_fname)
    # A couple of semi-hardcoded bits for this demo
    if D != 3
        println("Currently only 3D is supported")
        return
    end
    sys_size = parse.(it, SA["41", "41", "6"])
    println(sys_size, ", type is", typeof(sys_size))

    # prealocate pos
    pos = Vector{it}(undef, D)

    file = open(d_fname)
    line_i = 1
    while !eof(file)
        line = readline(file)
        set_pos!(line_i, pos, sys_size)
        line_i += 1
    end
end

function analyze_run_ini(s_fname, d_fname)
    # Read the ini file which has system size info
    setup = read(Inifile(), s_fname)
    sys = setup.sections["System parameters"]

    if D != parse(it, sys["DIM"])
        println("Currently only 3D is supported")
        return
    end

    sys_size_strings = SA[sys["size_x"], sys["size_y"], sys["size_z"]]
    sys_size = parse.(it, sys_size_strings)
    println(sys_size, ", type is", typeof(sys_size))

    # prealocate pos
    pos = Vector{it}(undef, D)

    file = open(d_fname)
    line_i = 1
    while !eof(file)
        line = readline(file)
        set_pos!(line_i, pos, sys_size)
        line_i += 1
    end
end

function set_pos!(i, pos, sys_size)
    D = length(sys_size)
    pos[D] = i
    for i in D:-1:2
        x, y = fldmod1(pos[i], sys_size[i])
        pos[i] = y
        pos[i-1] = x
    end
end

# Trigger any compilations
analyze_run_noini("setup.dat", "data.dat")
analyze_run_ini("setup.dat", "data.dat")

# And test
println("Starting benchmark runs")
@time analyze_run_noini("setup.dat", "data.dat")
@time analyze_run_ini("setup.dat", "data.dat")

this again has the analyze_run function duplicated so that the issue can be demonstrated. This can be directly ran in julia as “julia script.jl” and relies on two files being present - “data.dat” which can be any text file, it shouldn’t really matter and “setup.dat” which should look something like

[System parameters]
DIM = 3
size_x = 41
size_y = 41
size_z = 6

When I run these I get the function that actually loads the sys_size from the setup.dat file to allocate twice as many times as the one that does not. Given everything I now suspect this is something to do with type stability? But I struggle to find why/how and also how to fix it.

11

Another quick update, so turns out if I first declare sys_size and then set each of its elements using parse then I do not get these extra allocations. This works for me, however I would still like to be able to understand this. Does this mean that somehow the broadcasted parse call does not have a clear type or is there something else going on? And how does that lead to one additional allocation when I use set_pos!? I am very lost on why all of this is happening the way it is, especially since I have also noticed that if i replace the line_i argument in the set_pos! call then I do not get the extra allocations even when using the old version of defining sys_size.

12

EDIT: This likely does not apply here. See @stevengj below.

The function is likely not type-stable because Julia cannot deduce the type of sys_size from the type of it. It’s the value of it that determines the type!

Try maye:

function analyze_run_noini(s_fname, d_fname, index_type::Type{T}=Int64) where T
    ...
    sys_size = parse.(T, SA["41", "41", "6"])

Why do you use this strange construct with parse and SA?

13

Works for me:

julia> const it = Int64
Int64

julia> foo(x) = parse.(it, x);

julia> using Test

julia> @inferred foo(["41", "41", "6"]) # throws an error if type is not correctly inferred
3-element Vector{Int64}:
 41
 41
  6

But you can check with @code_warntype to be sure, or just explicitly write Int64 in the function to see if that changes things.

14

You are right. I don’t think it is type unstable.

I think I had a situation once, where I passed a type as an argument which lead to instability and that was then fixed by using the ::Type{T} ... where T incantation but this case is different. Should have checked before writing :slight_smile:

15

Hello again, I have tried using the @code_warntype macro (thank you for suggesting it I was not aware of it) and I think the code is actually unstable. When I use the macro on the original function I get a sys_size::Any, but with the version that doesn’t allocate it is the correct SVector. Given that as mentioned parse.(it, [“41”, “41”, “6”]) and parse.(it, SA[“41”, “41”, “6”]) both pass with @inferred I think the type instability must be coming from the IniFile library. It is very good to know that things like this can happen and how to debug them using the macros!

Finally a less important thing that originally puzzled me, when I use the original type unstable code but pass a numerical value to set_pos! as set_pos!(1, pos, sys_size) instead of the variable line_i, the extra allocations go away. Is this because of some optimization taking place?

Thank you everyone for helping me find the issue and correct the code!

16

When you read something from an unstructured file, probably it doesn’t know the type until runtime. If you know it will be an integer you can add a type assertion to fix it. e.g. sz = Int(sys_size)::Int.