Broadcast/map return type for a function returning a Tuple

Internals & Design
,
1

When a function returns a tuple, for example the value of the function and its derivative:

f(x) = x^2, 2x
y, ∂x = f(2.)

Would you expect this pattern to work with broadcast or map? For example:

y, ∂x = f.([2., 3.])
y, ∂x = map(f, [3., 4., 5.])

Unfortunately this doesn’t work. The return type is an array of tuples, not tuple of arrays. For my automatic differentiation library I had to code my own version of broadcast (called it multicast) that does returns a tuple. But I still miss the short syntax of .( and the chain / fuse broadcast performance improvements.

What do you think is the right behavior for broadcast / map from the user perspective?

2 Likes
2

I may be missing something …


f(x) = x^2, 2x
f(2.0)
# (4.0, 4.0)
typeof( f(2.0) )
Tuple{Float64, Float64}

If you map or broadcast over multiple Float64 arguments to f(), you should expect to get back some values, each of which has type Tuple{Float64, Float64}. Julia collects them as a 1-dimensional array.

You can get the result as a tuple of tuples, and keep the concise syntax:


> f(x) = x^2, 2x
f (generic function with 1 method)

> ( f.([1.0, 2.0, 3.0])... )
((1.0,2.0),(4.0,4.0),(9.0,6.0))
3

Unfortunately, it depends. Both an array of tuples and a tuple of arrays can make sense depending on the use case. The current definition of broadcast is the most consistent one, as it always returns an array with entries equal to the result of f. What you do is a little fancier, since you special-case tuples.

4

It is not the same to work with tuples of tuples or array of tuples when you keep constantly using different portions of your returns in different calculations. It could well be that derivatives are a special case when the forward pass keeps working on the values and then the backwards pass of the chain rule is working on the partial derivatives only. That’s why I’m asking if anybody else feels strongly about it.

5

FWIW, I think the current behavior is the right generic conception of what map should do in the sense that your proposed behavior is helpful in some recurring special cases, but would generally not make sense when applied to arbitrary functions. I think implementing your proposed semantics might even require making Julia’s type-inference system part of the official semantics of the language, which is, I believe, strictly off-limits as far as language proposals go.

To see why I believe your proposal breaks down, suppose that we know that f(x::Int) returns a Dict. Should the result of applying map(f, [1, 2, 3]) be an Array{Dict} or a Dict{Array}? The latter behavior would require much stronger invariants than the former: returning Dict{Array} would only make sense if the keys in the result of f(x::Int) were invariant with respect to all values of x.

You don’t even need to consider something weird like Dict to see how strongly your proposal depends upon implicit invariants. If f(x::Int) = ntuple(z -> z - 1, x), how would your new map function work? What would it do if the return type for f weren’t known to the type system? Even in your original example, the type system needs to know a lot about f before it can decide to return two arrays rather than one.

3 Likes
6

Wouldn’t an unzip function help? Then one could do

y, ∂x = unzip(f.([2., 3.]))

Unfortunately, there is no unzip in base. The following works reasonably well for me but I’m sure one can do better:

@generated function unzip{N}(it, ::Type{NTuple{N}})
    :(($((:([e[$i] for e in it]) for i = 1:N)...)),)
end
unzip(it) = unzip(it, eltype(it))
1 Like