Generic function for pairwise operations

General Usage
1

A very commonly used design pattern is to apply a function to all unique pairs of elements in a container, e.g.

f(x,y) = x*y
a = randn(5)
ret = eltype(a)[] #the example returns a vector, but in this case an UpperTriangular Matrix might be better.
for i in 1:(length(a-1))
   for j in (i+1):length(a)
      push!(ret, f(a[i], a[j]))
   end
end

(see e.g. Fast Numeric Computation in Julia for a use of pairwise), but it could also apply to all pairwise combination of columns or rows in a Matrix or higher-dimensional arrays.
Julia provides map for elementwise operations. Is there such a function for pairwise operations? Alternatively, if someone were to implement such a function, where would it be put? It should be so relatively general that it should be close to Base, I think to be really interesting, or perhaps somewhere like Iterators.jl?
Distances.jl has a pairwise function that implements some of this functionality efficiently, but AFAICS not all of it.
Thanks!

1 Like
2

I’m not aware of such functionality. My sense is that there’s a lot of useful abstractions that aren’t quite useful enough for someone to have built them, made them robust and decided to maintain them in the future.

In general, I very strongly encourage building all new functionality in a package that you both control and maintain. Base might be a good place for this, but it’s much better for everyone to start with functionality outside of Base.

On a sidenote, I’d want to know whether you want pairwise_map(f, M), which applies f to all pairs of entries in M, or whether you want map(f, pairs(x, y)), which takes two iterators and combines them into a single iterator that is mapped into a vector or other flat iterable.

1 Like
3

Ah, yes, I had a feeling the answer might be something like that. Well I definitely would find something like this useful so I might give it a stab sometime (or @ChrisRackauckas would it belong in GitHub - ChrisRackauckas/VectorizedRoutines.jl: Familiar vectorized routines from R/Python/MATLAB, plus some more. ?)

I think I’d want pairwise_map(f, M), perhaps with pairwise_mapslices(f, M) for multidimensional arrays. The return type is also tricky - maybe it should be an UpperTriangular (for linear containers)?

4

Yes, this is exactly the kind of stuff I’m gathering there if it has no other home!

1 Like
5

This sounds like you’re assuming that the function being applied pairwise is symmetric, no?

1 Like
6

argh, yes I was. Good point!

7

For cross-reference, the discussion continued here https://github.com/ChrisRackauckas/VectorizedRoutines.jl/issues/8 and there is now a PR here: https://github.com/ChrisRackauckas/VectorizedRoutines.jl/pull/9

8

@mkborregaard, your code doesn’t work… I get the following error with Julia 1.1.1:

MethodError: no method matching -(::Array{Float64,1}, ::Int64) Closest candidates are: -(!Matched::Complex{Bool}, ::Real) at complex.jl:298 -(!Matched::Missing, ::Number) at missing.jl:97 -(!Matched::Base.CoreLogging.LogLevel, ::Integer) at logging.jl:107 ... top-level scope at none:0

What is wrong?

9

I presume it was meant to be for i in 1:(length(a)-1). Since we’re here, note that this is the same as

[f(x,y) for (i,x) in enumerate(a), (j,y) in enumerate(a) if i>j]
3 Likes
10

Ah! Didn’t think about the array comprehesion! Good!