recent broadcast changes (iterate by default), scalar struct, and `@.`

Internals & Design
21

I consider the new broadcasting infrastructure one of the great strides made in Julia 0.7.

I use broadcasting extensively and have been very happy with the changes, but in upgrading my code have noticed that that while the default iteration behavior is generally most profitable, it causes me to litter my code with many Ref wrappers.

While not backwards incompatible, the upgrade path for broadcasting would seem to me to be greatly eased by the addition of a shorthand for Ref. Is there a chance of implementing the &x = Ref(x) sugar for 0.7 to help here?

9 Likes
22

Thanks for the discussion.
On the one hand, for a user the material would obviously (intuitively) not be iterable,
so there would be no readability issue with it being treated as “scalar”.
On the other hand, since broadcasting is something explicitly requested in julia,
it makes sense to issue an error if the argument does not support this operation.

There is a psychological point that might be worth emphasizing:
for numpy or octave users like me, broadcasting is never explicitly asked for.
Actually the @. macro, awesome for readability,
made me think in octave’s way,
forgetting that @. is requesting broadcasting.

The planned design is all about “thinking the julia way”, which seems good in the long run (consistency).

In this point of view,
chi1 = @. chi(&m1, energy)
would be fine.
There might be some confusion with the $ usage to avoid broadcasting on a function though.

But it will really require a good communication (like this discussion),
otherwise for a lot of potential users, it will be hard to accept that the default is not
broadcastable(x) = Ref(x)
[iterables having in their interface “Required methods” an appropriate standard broadcastable method].

1 Like
23

Would an issue about the & syntax be welcome ?

1 Like
24

After checking the “broadcasting” issues and the git logs, filed
https://github.com/JuliaLang/julia/issues/27563

@mbauman By the way, having read more about this, thanks for your amazing work !

4 Likes
25

I was partly convinced:

But this is true only for the single scalar argument case, because for multiple arguments,
requesting a broadcast between a scalar and an iterable is a valid operation.

1 Like
26

The initial issue is still there in the just released 0.7.

And in Version 1.0.0-rc1.0 (2018-08-07 16:29 UTC):

struct Material
    a::Float64
end
chi(m::Material, energy) = m.a / energy

m1 = Material(1.0)
energy = 1.5:0.5:2.5
chi1 = @. chi(m1, energy)

ERROR: MethodError: no method matching length(::Material)

chi1 = @. chi(&m1, energy)
ERROR: syntax: invalid syntax &m1

There has been a lot of progress on the broadcasting,
but this is a particularly tricky question, with many possible points of view,
tests/feedback from more users seem desirable before 1.0.

27

I would recommend defining broadcastable(x::Material) = Ref(x).

This is indeed a tricky issue, and I do lament that it makes some things more awkward… particularly if you don’t have control over the package that defines Material. But at the same time it’s a trade-off aimed at simplifying our rules.

I will note that there were very qualified and highly respected developers on the opposite side of the debate before my fix — developers who would have been able to propose a reversion or alternative or just continue pushing back before the tagging of 0.7. The group involved in the discussion was even prompted for an update at a point when there still would have been time to reconsider: https://github.com/JuliaLang/julia/issues/18618#issuecomment-405267705.

2 Likes
28

This is my single biggest disappointment with v0.7/v1.0, which has otherwise brought me delight.

Like others, I broadcast with structs all the time. Having to put broadcastable(x::MyType) = Ref(x) for every single new type which I want to treat this way is tedious and requires one to know in advance that a user will want to do this. To be safe, I should therefore write that for every single struct I ever expose just in case, which in practice means any types which aren’t supposed to iterate. (It may be insightful to note that as a ‘user’ of the language who maintains a few packages, I have never written an iterable type.)

An abstract example below is perfect broadcast territory:

julia-0.7> struct A end

julia-0.7> struct B
               x::Int
           end

julia-0.7> doit(a::A, b::B) = 2*b.x
doit (generic function with 1 method)

julia-0.7> a = A()
A()

julia-0.7> bs = [B(i) for i in 1:3]
3-element Array{B,1}:
 B(1)
 B(2)
 B(3)

julia-0.7> doit.(a, bs)
┌ Warning: broadcast will default to iterating over its arguments in the future. Wrap arguments of
│ type `x::A` with `Ref(x)` to ensure they broadcast as "scalar" elements.
│   caller = ip:0x0
└ @ Core :-1
3-element Array{Int64,1}:
 2
 4
 6

…which to avoid requires the original author writing a broadcastable method for all public types.

I appreciate very much that there may have been other discussions. My reading of #18618 is simply that nobody managed to stop this happening, rather than there being support for it.

The amazing broadcasting syntax has now become less useful to me, which is a sad thing.

Despite all that, here’s to a successful v1.0!

7 Likes
29

From my user point of view, this neither simpler nor more systematic than:
“things are treated as scalars,
unless their broadcastable method has been defined to something more useful”
The behavior is already entirely defined by this method. This is very clear [good job !].

Sure, this has been discussed at length.
This is probably one reason why there isn’t any feedback from core developers,
only from newcomers.
Another reason is that we are talking about the default broadcastable method, very shallow,
it does not change anything to the bulk of the broadcasting mechanism.

Anyway, I left a comment on github, hopefully to help clarifying the arguments.

1 Like
30

Another similar use case example.
And another one.

1 Like
31

Another use case that bit me today:

julia> eachmatch.(r"a\da", ["a1a2a3a", "a4a5a6a", "a7a8a9a"])
ERROR: MethodError: no method matching length(::Regex)
Closest candidates are:
  length(::Core.SimpleVector) at essentials.jl:571
  length(::Base.MethodList) at reflection.jl:728
  length(::Core.MethodTable) at reflection.jl:802
  ...
Stacktrace:
 [1] _similar_for(::UnitRange{Int64}, ::Type, ::Regex, ::Base.HasLength) at ./array.jl:532
 [2] _collect(::UnitRange{Int64}, ::Regex, ::Base.HasEltype, ::Base.HasLength) at ./array.jl:563
 [3] collect(::Regex) at ./array.jl:557
 [4] broadcastable(::Regex) at ./broadcast.jl:609
 [5] broadcasted(::Function, ::Regex, ::Array{String,1}) at ./broadcast.jl:1139
 [6] top-level scope at none:0

julia> eachmatch.(Ref(r"a\da"), ["a1a2a3a", "a4a5a6a", "a7a8a9a"])
3-element Array{Base.RegexMatchIterator,1}:
 Base.RegexMatchIterator(r"a\da", "a1a2a3a", false)
 Base.RegexMatchIterator(r"a\da", "a4a5a6a", false)
 Base.RegexMatchIterator(r"a\da", "a7a8a9a", false)

Perhaps broadcastable for Regex has been forgotten? It’s not like one would ever intend to iterate over a regex, right? I don’t have access to my GitHub account right now, so no PR from me for the moment.

Another problem I see: hadn’t I followed this thread/discussion, I would not have known how to fix it.

In general, there may be many more types in Base with a similar behaviour.

3 Likes
33

Why was Ref used for this, instead of defining a dedicated type to act as a wrapper, with a name like Scalar, which is more explicit? Even if internally Scalar behaved identically to Ref. I don’t see how the semantics of Ref (C Interface · The Julia Language) apply in this context.

Also, looking at something like broadcast(Scalar(x), y) tells you immediately what it does, whereas broadcast(Ref(x), y) probably takes some getting used to.

What’s the reasoning here?

4 Likes
34

Probably that Ref already exists and behaves that way, so that no change was needed for it to work? A similar type with a more intuitive name can always be added later without breaking anything.

1 Like
35

Yup, it was in the name of simplicity that I chose Ref. I actually started out with a:

struct Scalar{T} <: AbstractArray{T, 0}
    val::T
end
Base.getindex(s::Scalar) = s.val
Base.size(::Scalar) = ()

but ended up choosing Ref to primarily avoid exporting a new name at a time when we were trying to reduce the “breakable” surface of Julia. You can always do const Scalar = Ref.

36

It seems that Ref(x) always results in allocation:

@btime Ref(x) setup=(x=1)

gives

4.026 ns (1 allocation: 16 bytes)
Base.RefValue{Int64}(1)

Is this true? Is there a way to avoid the allocation?

1 Like
37

If you return a Ref then it has to be allocated… But that’s not what you want to do here.

julia> f(x,y) = Ref(x) .+ Ref(y)
f (generic function with 1 method)

julia> using BenchmarkTools

julia> @btime f(1,2)
  1.507 ns (0 allocations: 0 bytes)
3

julia> @code_llvm f(1,2)

; Function f
; Location: REPL[8]:1
define i64 @julia_f_36898(i64, i64) {
top:
; Function materialize; {
; Location: broadcast.jl:724
; Function copy; {
; Location: broadcast.jl:734
; Function getindex; {
; Location: broadcast.jl:507
; Function _broadcast_getindex; {
; Location: broadcast.jl:547
; Function _broadcast_getindex_evalf; {
; Location: broadcast.jl:574
; Function +; {
; Location: int.jl:53
  %2 = add i64 %1, %0
;}}}}}}
  ret i64 %2
}
4 Likes
38

I’m necro-ing this thread because I opened https://github.com/JuliaLang/julia/issues/28961, and I feel it would be better to have the “philosophical” discussion collected in one place. To be clear; I am discussing the philosophy of broadcasting, completely independent of any @. concerns. This is purely about what .( should “mean”. I’ll state my thesis first, then respond to the most pertinent comments from Jeff and Matt above.

I think the problem is that broadcasting is supposed to split its behavior depending on whether something is a “scalar” or not. In my opinion, this should be a binary decision; something is either a scalar or not. Flipping that logic, I would rather state something is either a container or it is not. The .( operator applies the “map operation over elements of a container” operation, with the escape hatch of operating upon a scalar as a single-element container. I believe that escape hatch is very important as it makes writing recursively-broadcast methods much easier (for example, mapping elements of tree structures).

With this mindset, we have to decide how to define something as a “scalar”. So far in Julia land, we’ve pretty much done this through multiple dispatch; call length(x), or iterable(x) or something like that. As Jeff notes above, this gives the problem that a new package bringing in new method definitions could cause “action at a distance” and suddenly change the broadcasting behavior for my type. We could argue about whether or not that is a feature or a bug, but since we are talking about the fundamental capabilities of a datatype, I think this is the kind of thing that should be solved by something tied to the datatype definition; e.g. a trait. Everything that is not marked Iterable is treated as a scalar.

With that stated, here are my responses to the two comments from May 29th:

The first problem is that it’s useless for f.(x) to be equivalent to f(x) — you bothered to write a dot, so we ought to do something with it.

I would agree with this if there weren’t so many examples precisely where f.(x) = f(x). IntX, FloatX, Symbol, Char, Ptr{X}, etc… there are many examples where this is the case, and I argue that this is a good thing; otherwise we should throw an error on f.(1). It’s difficult to write generic code that should work on both containers and scalars without this fundamental capability.

It’s also conceivable that a type could add or remove an iterate method, and that would silently change the behavior of your program.

I agree this is a serious concern; this is why I think we need a way to tie dispatch to “intrinsic information” about a datatype, rather than methods that can always be added later. I do not think that allowing package B to change whether a datatype defined in package A is Iterable or not is really something we desire; we would want the datatype definition itself to define whether it is Iterable through some kind of trait, and thus we could use that same declaration here.

By defaulting to errors for non-iterables, it’s a great big signal for library creators that they can opt-in to acting like a scalar if it is appropriate

Conversely, by defaulting to treating everything as a scalar, it allows all code to “run”, and if the library author actually wants something to be Iterable it is stated within that package. I don’t see why throwing an error is ever desirable in this case.

End-users can opt-in to scalar-like behaviors by wrapping in another broadcastable container

Wrapping in Ref() is a good solution for disabling broadcasting, I agree.

Were we to default to treating everything like a scalar, that means that they would have to collect an iterable into a different broadcastable structure in order to iterate over it.

I think you’re saying here to treat everything as a scalar, but instead I want to treat everything that does not contain the Iterable trait, which is slightly different. There would be no way to “opt-in” to broadcasting, as that requires explicit support, but there is always a way to opt-out, which already exists; Ref().

Even with base itself, this led to lots of surprises — some folks wanted to iterate over sets but treat dicts as scalars. Or broadcasting the array-ish Linear Algebra I object was quite a surprise.

All of this is solved by being consistent and supporting the Ref() disabling of broadcasting.

5 Likes
39

I’m in complete, absolute agreement.

The problem is that needing a separate configuration thingy leads to bugs. It means the default fallback is wrong the moment you define iterate but not the Iterable trait. This leads to bugs and inconsistencies. Sure, you can say that we need to be consistent, but if we ever get this wrong then we’re locked in for a major release cycle as this really is a fundamental behavior. There were many such examples in 0.6: Broadcast had one job (e.g. broadcasting over iterators and generator) · Issue #18618 · JuliaLang/julia · GitHub.

The other problem with changing the default to scalar instead of container is that it means that users cannot opt back into container-like semantics at a call site. This is that asymmetry I refer to above.

With respect to the broadcasting of Exprs — the issue that brought you here in the first place — I don’t think it’s clear what that should do. Does it broadcast over the arguments? Or does it treat the Expr as a scalar? An error seems sensible until someone makes a strong case for it to behave one way or another and either explicitly enables iteration or a scalar-like broadcastable definition.

There’s actually a new solution now that wasn’t available with the deprecations in 0.7: we could check applicable(iterate, x). IIRC, that needs an optimization to really work properly, but that would simply change an error condition to a new feature/behavior… and that could be done in a minor release.

3 Likes
40

No, because 1 is actually iterable and 0-dimensional; it supports the full broadcastable interface.

+1.

Requiring defining a trait for things to work is really annoying. I also think getting an error is better than doing something strange by default. Writing string.(x) and getting string(x) is confusing and looks like a bug. Functions should do things; they shouldn’t just ask their arguments, “Well, what do you want me to do? No answer? Ok I’ll do nothing then.”

In any case I don’t regret this decision.

2 Likes
41

I may be mixing things up here, but wouldn’t this whole debate be solved by the (iirc already planned) addition of enforcable interfaces? That way there’s no ambiguity whether something is an iterable or not - it would be implicit because of the defined interfaces. One way to achieve a similar effect right now is using applicable, as suggested above.

I may have understood the problem wrong though.