Weird type errors when manipulating arrays of functions

General Usage
1

Here are four expressions that I expect to evaluate to [sin, sin ∘ sin]:

accumulate(∘, [sin, sin])                    #1
accumulate(∘, [x -> sin(x), x -> sin(x)])    #2
accumulate(∘, [sin for i=1:2])               #3
accumulate(∘, [x -> sin(x) for i=1:2])       #4

However, only the second version is accepted by Julia. For the others, I get the following cryptic errors:

julia> accumulate(∘, [sin, sin])  #1
ERROR: MethodError: Cannot `convert` an object of type typeof(sin) to an object of type getfield(Base, Symbol("##52#53")){typeof(sin),typeof(sin)}

julia> accumulate(∘, [sin for i=1:2])  #3
ERROR: MethodError: Cannot `convert` an object of type typeof(sin) to an object of type getfield(Base, Symbol("##52#53")){typeof(sin),typeof(sin)}

julia> accumulate(∘, [x -> sin(x) for i=1:2])  #4
ERROR: MethodError: Cannot `convert` an object of type getfield(Main, Symbol("##100#102")) to an object of type getfield(Base, Symbol("##52#53")){getfield(Main, Symbol("##100#102")),getfield(Main, Symbol("##100#102"))}

Does anyone have a clue what’s happening here?

2

If I remember correctly, accumulate has some difficulties in determining the proper return type, so it’s constructing an overly narrow array as the result and then having trouble assigning the differently-typed function into slot 2. Simon Byrne has a PR that should fix it.

All four work if you use Any[…] or Function[…] arrays as input — #2 is the already the latter.

3

Thanks a lot!

4

Also, the getfield(...) is a closure.

I guess its printing could be improved, but could not find an issue, is there one?

5

As a new user of Julia coming from a functional programming background, I find several things surprising here. I do not have enough experience to judge whether or not they should be regarded as issues though.

First, why would closures be treated any differently than standard functions by the type system?

Then, why aren’t closure types stable under composition? For example:

julia> typeof(x -> x+1)
getfield(Main, Symbol("##33#34"))

julia> typeof((x -> x+1) ∘ (x -> x+1))
getfield(Base, Symbol("##52#53")){getfield(Main, Symbol("##39#41")),getfield(Main, Symbol("##40#42"))}

I could get an arbitrarily long type by composing more closures together.

6

I don’t think they are treated differently. You may want to read the development docs, eg the chapter on functions.

Because they generally aren’t? Eg

julia> c = x -> x isa Char ? x - 'a' : x + 'a'
#35 (generic function with 1 method)

julia> c('b')
1

julia> (c∘c)('b')
'b': ASCII/Unicode U+0062 (category Ll: Letter, lowercase)
7

Every generic function has its own type. Since Julia does type-based inference and specialization, this is precisely what makes higher order functions fast. It means that Julia can reason about the return types of functions that get passed as arguments (and a whole host of other optimizations). This wasn’t always the case… and changing it to be this way was a HUGE improvement!

2 Likes
8

@Tamas_Papp To me, one sign of closures and functions being treated differently by the type system is that the two following lines are not equivalent:

accumulate(∘, [sin, sin])                     # Type error
accumulate(∘, [x -> sin(x), x -> sin(x)])     # Ok

Also, I should have been more precise when I said that I expected “the type of closures to be stable under composition”. I referred to the specific case where the closure has one argument and where its input and output types coincide.

Edit: @mbauman answered my question.

9

A bit more directly, [sin, sin] is an array that can only hold typeof(sin) objects:

julia> A = [sin, sin]
2-element Array{typeof(sin),1}:
 sin
 sin

julia> A[1] = cos
ERROR: MethodError: Cannot `convert` an object of type typeof(cos) to an object of type typeof(sin)

julia> A[1] = x->sin(x)
ERROR: MethodError: Cannot `convert` an object of type getfield(Main, Symbol("##24#25")) to an object of type typeof(sin)

The array [x->sin(x), x->sin(x)] is actually holding onto two different functions; the two anonymous functions are not the same, so they have different types, so constructing an array with them chooses the narrowest common supertype: Function:

julia> B = [x->sin(x), x->sin(x)]
2-element Array{Function,1}:
 getfield(Main, Symbol("##30#32"))()
 getfield(Main, Symbol("##31#33"))()

julia> B[1] == B[2]
false

julia> B[1] = cos
cos (generic function with 12 methods)

julia> B
2-element Array{Function,1}:
 cos
 getfield(Main, Symbol("##31#33"))()

Were you to construct this such that you have the same anonymous function, it’ll behave like A:

julia> g = x->sin(x); C = [g, g]
2-element Array{getfield(Main, Symbol("##34#35")),1}:
 getfield(Main, Symbol("##34#35"))()
 getfield(Main, Symbol("##34#35"))()

julia> C[1] = sin
ERROR: MethodError: Cannot `convert` an object of type typeof(sin) to an object of type getfield(Main, Symbol("##34#35"))
2 Likes