Details of the ArrayAndChar broadcasting example

The Julia manual uses a type ArrayAndChar as its example of how to customise broadcasting.

Following a suggestion from @mcabbott, I’m implementing this for real. I’m running into a bunch of complications that the manual doesn’t discuss, and I was hoping someone could point me to a fully implemented array subtype that is similar to this example.

My type is actually ArrayAndTensorShape instead of ArrayAndChar, but that is a minor detail.

I have found two major complications so far.

Firstly, a function might be broadcast over multiple ArrayAndChar arguments. Suppose I want to do the obvious thing if the chars are the same, and throw an error if they differ. Should I tinker with the rather brittle find_aac function in the manual, or is there a neater and more reliable way that puts the char into a special subtype of BroadcastStyle and specializes the two-argument BroadcastStyle constructor?

Secondly, view types. Let’s say I have a method

return_char(A::ArrayAndChar) = A.char

I want things like return_char(Diagonal(A)) and return_char(reshape(A', ...)) to work, even though the view types might be nested arbitrarily deeply. I can see two ways of doing this.

The first way is a kludge:

const ChildArray = 
        Ts = [m.sig.parameters[2] for m in methods(parent)]
        filter!(T -> T != AbstractArray, Ts)

return_char(A::ChildArray) = return_char(parent(A))

That solves the immediate problem, but what about A .+ B' where A and B are ArrayAndChar?

The second way puts the view type on the inside, along the lines of

for T = [m.sig.parameters[2] for m in methods(parent)]
    @eval ($T)(A::ArrayAndChar, args...) =
        ArrayAndChar($T(, args...), A.char)

That seems like a total hack, whose compatibility with the Julia interpreter would have to be very carefully managed. Also, how will A*B know to use the customized method *(::Diagonal, ::Diagonal) when and are Diagonal?

First, I know of at least one package that has a more serious variant of the ArrayAndChar example, ImageMetadata.jl. It’s possible that some of these issues are resolved there.

Should I tinker with the rather brittle find_aac function in the manual, or is there a neater and more reliable way that puts the char into a special subtype of BroadcastStyle and specializes the two-argument BroadcastStyle constructor?

You could presumably do either one. If you do the latter, you can look at base/broadcast.jl to see how the AbstractArrayStyle variants are handled. I’d avoid making the char a type-parameter, though, to prevent a heavy compile time burden.

For the view-type problem you describe, the ChildArray kludge is definitely a kludge, and will introduce all sorts of code-loading-order problems in addition to not really fixing the problem. I’ve tended to go your second route, but never with your attempt to autogenerate the code; I would be scared to see what weird corner cases I hadn’t thought of and what other constructor types I would have missed. (For example, a PaddedView takes the fill value as the first argument.) Typically I manually write out all the cases I care about, with the list generally growing over time. This isn’t a great solution, and while I’ve never really sat down and thought hard about the problem, nothing better has quickly come to mind.


There is a neat solution for part of what you ask about wrapper types, which I learned from @oxinabox. This is to exploit the fact that they all define parent, while parent(A)===A for anything else. So you can recursively unwrap, giving up if the type stops changing. And this should be zero-cost.

return_char(A::ArrayAndChar) = A.char
function return_char(A::AbstractArray)
    P = parent(A)
    typeof(P) === typeof(A) ? nothing : return_char(P)

But making * aware of all arbitrarily deeply nested subtypes which contain your type, I don’t think there is a good solution for for that. You can define methods for some list of wrappers outside of yours, but as the pile gets higher it gets increasingly fragile. If * (etc.) used some kind of trait system like #25558 (but perhaps encoding things other than the strides) then it might be easier to hook such things on.

For broadcasting I have nothing very useful to say, but if you’d like an example I got this working at some point, and then forgot how.

1 Like

Thanks. It looks like ImageMetadata takes that approach too. E.g., it handles adjoint, but not transpose or Diagonal. It’s good to know that I’m not missing an obvious better way.

In an ideal world, I think that AbstractArray would have subtypes. There would be JustAnArray for structured matrix types T that represent a subset of Array, and for which T∘Array is the identity mapping. The other type would be CanBeTreatedAsAnArray. This might an image that exposes its pixels as an array of colors, but has non-pixel metadata that gets lost when it is cast to Array.

I can dream about a broadcasting mechanism that lowers everything to JustAnArray, does one BroadcastStyle reduction to find the appropriate structured type to store the result, then does a second reduction of CanBeTreatedAsAnArray to find a prototype A for similar(A, S <: JustAnArray{T}, T, dims).

1 Like

Some combination of traits and types might indeed make this possible. There may well be a very good solution to this problem, and it’s just a question of carving out the time to think about it more deeply. I’m glad you’re thinking about this, and if you do start assembling a solution I’m sure there will be a number of people interested in it.


As a first step, I’ll try AnnotatedArray{M,S,T,N}, where M is an arbitrary metadata type, and S is a structured array type with eltype T. Then I can think about what parts of the implementation could be generalised as broadcast machinery.