Hi, I need to analyze a long vector and calculate the occurrences of certain letters, as it will be very long it will be better to use parallel processing for this. The thing is, I can do this with map(), storing tasks. But I would like to compare the efficiency in case of just using “for” in this…

If you want to save tasks from the for loop, you need to allocate an array and push the tasks into it, e.g.: using ChunkSplitters array1 = repeat([1,2],1111) chk = chunks(1:length(array1); n=Threads.nthreads(), split=:batch) counts_total = Dict(ii => 0 for ii in [1,2]) tasks = Task[] for inds i…

Another general tip: A Dict has constant time membership access but it is rather slow. If you want to count symbols and there is a limited (known) amount then it will likely be much faster if you store the counts in a Vector{Int} that you index by the element somehow.

@Satvik thank you for you attention and support. The code worked really well. @abraemer Thanks for the tip, from what I saw there’s no way to put names in vectors, right? I’m going to think of some way to index numerically and create a fixed dictionary like: A → 1, B → 2, C → 3 and so on… Here is…

[image] felipe_1: going to think of some way to index numerically and create a fixed dictionary like: A → 1, B → 2, C → 3 and so on… Just to be sure: You don’t mean to have a Dict that does this mapping right? Because that would defeat the purpose ofc. You need to to that yourself somehow. I…

[image] abraemer: The benchmarks you posted show quite a lot of allocations. I wonder why that is, could you include the code of the functions as well? The code uses array1[inds], which makes a copy. If you add a @view here, the size of the allocations is vastly reduced: julia> array1 = repe…

You need to to that yourself somehow. If you have an ASCII string you could simply take the char value to index a Vector of size 255 for example Yes that was my idea without Dict…I will implement indexing in the vector and show how it turned out here. but I didn’t understand what you suggested n…

Dear @eldee Thank you very much for the tips and implementations, I will be able to test it soon on a computer where I can allocate ± 50 threads and I will show the result here. thank you my best

@abraemer I made the changes you suggested. @eldee I implemented your adjustments along with the suggestions above. Performance is now much better after your suggestions. Thank you very much! Here is the updated code and benchmark. function count_loop2(values, vec_reference) chk = chunks(1:l…

Some comments: [image] felipe_1: repeat([0],length(vec_reference)) This (and similarly below) should be zeros(Int, length(vec_reference) [image] felipe_1: findall(x->x==x, vec_reference) I don’t think this does what you intend? Since x==x is always true (except for NaN I think) you t…

FWIW, if you would want to write this with OhMyThreads.jl, this would be one (simple but not optimal) way to do it: using OhMyThreads using StaticArrays function count_loop_omt(values, vec_reference) @tasks for v in values @set reducer = + @local subcounts = @MVector zeros(Int,…

[image] abraemer: [image] felipe_1: cur_id = findall(x->x==val, vec_reference) Here you should use findfirst(x->x==val, vec_reference) to just get first index (which is also the only index). `` Or better yet, keep vec_reference sorted, and use searchsortedfirst.

Manage tasks from Threads.@spawn without using map()

General Usage

sgaure August 24, 2024, 7:58am 12

Or better yet, keep vec_reference sorted, and use searchsortedfirst.

Help needed getting started with threads!

Topic		Replies	Views
No more threadid indexing? [thread-local storage] Performance multithreading , threads	36	1378	August 15, 2025
Simple Parallel Examples for Embarrassingly Simple Problems Julia at Scale	29	7574	April 23, 2021
How to execute tasks in parallel in a for loop Performance parallel , multithreading , juliapro , optimization	27	2516	November 29, 2023
More threads, slower code, even if not spawning them Performance	19	918	January 29, 2022
Parallel reductions Julia at Scale	23	5385	June 19, 2022

Manage tasks from Threads.@spawn without using map()

Related topics