Segmentation fault in multi thread code

New to Julia
1

Hello, I would appreciate some advice on a multi thread code causing a segmentation fault with my workstation Dell Inc. Precision 7960 Tower (Intel® Xeon® w9-3495X × 112), running Ubuntu 24.04.2 LTS. Here you find a simplified situation. I have a matrix, which I update inside a multi thread for cycle. I have also divided the total number of samples to process into chunks.

# classify the trajectories. parallel version 
m = size(orbit_array,1)
output_report = zeros(m,6)
num_threads = Threads.nthreads() #I use 32 out of 112
chunk_size = 100
start_num = 1
selected_elements = range(start_num,m)
# Total number of elements to process

total_elements = length(selected_elements)

# the orbits are processed in chunks. At the end of the chunk, results are saved
num_chunks = ceil(Int, length(selected_elements) / chunk_size)

# Calculate chunk size (total elements divided by the number of chunks)
chunk_size = ceil(Int, total_elements / num_chunks)

# Create a list of ranges for each chunk
chunks = [(chunk_id - 1) * chunk_size + 1: min(chunk_id * chunk_size, total_elements) for chunk_id in 1:num_chunks]

npzwrite("output.npy", output_report)

for chunk_id in 1:num_chunks
    # Get the range of elements for this chunk
    range_start, range_end = chunks[chunk_id][1], chunks[chunk_id][end]

    # Load intermediate results
    output_report = npzread("output.npy")

    @threads for i in range_start:range_end
        j = selected_elements[i]
        println("   Thread $(threadid()). orbit $j, element $i of $total_elements")

        output_report[j,6] = classifier(orbit_array[j,:,:])
    end


    println("Finished chunk $chunk_id, saving intermediate results...\n")
    npzwrite("output.npy", output_report)
end

The classifier function just provides a float number, which I add at the last column at the corresponding row. The classifier (a complex function) is not important in this discussion: the error arises also when substituting it with a simpler output_report[j,6] = 1.

When running this code on VSCode a segmentation fault is produced on this machine. I am not able to report the error output, as the julia terminal crashes right after. I have also tried another version of this code by using the @spawn method, but the result is the same. However, whenever I run this code on my Windows laptop, everything works as expected, without bugs.

What can be the issue with the workstation? Thank you for your help.

2

I think it’s very unlikely to crash.
either add a local so that local j = selected_elements[i],
or add a @lock to output_report[j,6] = classifier(orbit_array[j,:,:]).

Otherwise I cannot figure out a reason to crash—you didn’t give the runnable code. And the only multithreading part is the @threads for.

3

Hi, and welcome to the Julia community!

What happens if you run the code (or MWE thereof) in a different terminal than the VS Code one?

I’m not sure that this would cause a segmentation error, but note that you’re (potentially) allocating a lot of memory here, as orbit_array[j, :, :] this makes a copy for each task. You could see if @view(orbit_array[j, :, :]) fixes the problem. (Also, if possible, consider reordering the dimensions of orbit_array so that you get orbit_array[:, :, j], which is a contiguous chunk of memory.)

4

Hmm, I can’t see anything here which could cause a segfault, only several things which will slow things down.

Could it be that npzread/npzwrite does something dubious? Could you try serialize/deserialize instead (with using Serialization).

And what’s the output of versioninfo()?