Fastest approach to returning a vector of elements choose 2

General Usage Performance
21

Makes sense. I tried threading what I posted before with the choose2 generator that @abraemer posted, but the threading doesn’t like the generator:

TaskFailedException

    nested task error: ArgumentError: Iterates of the argument to Flatten are not known to have constant length
    Stacktrace:
     [1] flatten_length(f::Base.Iterators.Flatten{Base.Generator{UnitRange{Int64}, var"#993#994"{Vector{Line{2, Float32}}}}}, T::Type)
       @ Base.Iterators .\iterators.jl:1185
     [2] length(f::Base.Iterators.Flatten{Base.Generator{UnitRange{Int64}, var"#993#994"{Vector{Line{2, Float32}}}}})
       @ Base.Iterators .\iterators.jl:1187
     [3] (::var"#402#threadsfor_fun#997"{var"#402#threadsfor_fun#996#998"{Vector{Tuple{Line{2, Float32}, Line{2, Float32}, Point{2, Float32}}}, Base.Iterators.Flatten{Base.Generator{UnitRange{Int64}, var"#993#994"{Vector{Line{2, Float32}}}}}}})(tid::Int64; onethread::Bool)
       @ Main .\threadingconstructs.jl:132
     [4] (::var"#402#threadsfor_fun#997"{var"#402#threadsfor_fun#996#998"{Vector{Tuple{Line{2, Float32}, Line{2, Float32}, Point{2, Float32}}}, Base.Iterators.Flatten{Base.Generator{UnitRange{Int64}, var"#993#994"{Vector{Line{2, Float32}}}}}}})(tid::Int64)
       @ Main .\threadingconstructs.jl:130
     [5] (::Base.Threads.var"#1#2"{var"#402#threadsfor_fun#997"{var"#402#threadsfor_fun#996#998"{Vector{Tuple{Line{2, Float32}, Line{2, Float32}, Point{2, Float32}}}, Base.Iterators.Flatten{Base.Generator{UnitRange{Int64}, var"#993#994"{Vector{Line{2, Float32}}}}}}}, Int64})()
       @ Base.Threads .\threadingconstructs.jl:108

Is it possible to thread a generator like that without having to just collect it?

intersections = Vector{Tuple{GeometryBasics.Line{2, Float32}, GeometryBasics.Line{2, Float32}, GeometryBasics.Point{2,Float32}}}() 
  
Threads.@threads for (seg1, seg2) in choose2_generator(segments)  
    x1, y1, x2, y2, x3, y3, x4, y4, t, u = line_data1(seg1, seg2,)
    
        if is_intersecting(t, u)
            point = lineintersection(x1, y1, x1-x2, y1-y2, x3, y3, x3-x4, y3-y4, t, u)
            if point != false && not_on_endpoint(seg1, seg2, point)
    
                push!(intersections, (seg1, seg2,  point) )
            end
   
        end
            
    end
22

I don’t think you can parallelize construction of a generator. For your situation, where you don’t just want the n-choose-2 pairs, but you need to do significant calculations to determine which to keep, you probably don’t want a generator. You should write functions to do your selection as rapidly and with as few allocations as possible, and then just include them in a loop that you multithread. That’s my take, perhaps others here have better ideas.

1 Like
23

Because the generator requires that you iterate it sequentially, and @threads doesn’t do that.