# Broadcasting - Mathematical notation

**URL:** <https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327>\
**Category:** Offtopic\
**Tags:** math\
**Created:** [October 6, 2021, 7:14pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327 "2021-10-06T19:14:10Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)\
**Post date:** [October 6, 2021, 7:14pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/1 "2021-10-06T19:14:10Z")

</div>

What’s the proper notation for something like this?

```julia
x = [0.09, 0.2, 0.05 ,0.01]
w = [0.81 0.92 0.86 0.32; 0.42 0.95 0.96 0.53]
b = [0.57, 0.52]
f(x) = σ.(b + w * x)

```

f(x) = \sigma.(b + wx)

Is there a common notation that I can just plug into the spot where the dot is or do I need something entirely different?

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 6, 2021, 7:30pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/2 "2021-10-06T19:30:23Z")

</div>

Not entirely sure I understand the question, but, presumably, the thing that goes where the dot is, is, well, a _dot_.

I mean, doesn’t it already work? You should probably add a dot to the plus sign as well, though.

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [October 6, 2021, 7:31pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/3 "2021-10-06T19:31:35Z")

</div>

In my experience, conventions are not always consistent across fields. [This](https://math.stackexchange.com/questions/52578/symbol-for-elementwise-multiplication-of-vectors) might be helpful. If in doubt, define the notation for your readers.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 6, 2021, 7:34pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/4 "2021-10-06T19:34:32Z")

</div>

Oh, you were asking how to denote elementwise application in _maths notation_? I didn’t get that.

---

<div class="post-metadata">

**Author:** ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)\
**Post date:** [October 6, 2021, 7:35pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/6 "2021-10-06T19:35:37Z")

</div>

> Oh, you were asking how to denote elementwise application in _maths notation_ ? I didn’t get that.

Yes, sorry I wasn’t clear on that. f(x) just applies the logistic/sigmoid function to each element in the 2 x 1 vector that is the result of b + wx. In Julia code, it’s clear to me that I’m applying sigmoid to each element of the vector (because of the dot) but I’m trying to write a small presentation for non-Julians that explains the code and I couldn’t find a standard way to notate that operation…

---

<div class="post-metadata">

**Author:** ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)\
**Post date:** [October 6, 2021, 7:47pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/8 "2021-10-06T19:47:50Z")

</div>

> [@Christopher\_Fisher](#):
>
> In my experience, conventions are not always consistent across fields. [This](https://math.stackexchange.com/questions/52578/symbol-for-elementwise-multiplication-of-vectors) might be helpful. If in doubt, define the notation for your readers.

Since this is just for an internal (work) audience, I’m going to just define my own notation 🙂

I hereby declare that \star indicates the element-wise application of a function to a vector.

f(x) = \sigma \star(b + wx)

because it just looks cool 🕶

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 6, 2021, 7:50pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/9 "2021-10-06T19:50:02Z")

</div>

I just checked with my fellow mathematicians, and there is no standard notation for that.

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [October 6, 2021, 7:57pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/10 "2021-10-06T19:57:31Z")

</div>

Check this [post](https://math.stackexchange.com/a/3410631), and take `W(x) = b + w*x`

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [October 6, 2021, 8:16pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/11 "2021-10-06T20:16:38Z")

</div>

Mostly I (and I think people in general) think of broadcasting as a computational convenience.  
But it shows up in a suprisingly common bit of calculus (though I might be out of touch with what people do with calculus).  
The `pullback` of `y=sum(f, xs; dims)` is to broadcast the pullback of `f` at each element of `xs` against `ȳ`.  
This broadcasting is exactly what is needed to handle the different dimensionality of of `xs` and `ȳ` if the `dims` argument use used.

See this [code](https://github.com/JuliaDiff/ChainRules.jl/blob/v1.11.0/src/rulesets/Base/mapreduce.jl)

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 6, 2021, 8:18pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/12 "2021-10-06T20:18:31Z")

</div>

Broadcast multiplication is sometimes called the [Hadamard Product](https://en.wikipedia.org/wiki/Hadamard_product_(matrices)).

I don’t think there’s any general, well known mathematical notation for our full broadcasting semantics though

---

<div class="post-metadata">

**Author:** ![LeePhillips](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/leephillips/32/205514_2.png) [@LeePhillips](https://discourse.julialang.org/u/LeePhillips)\
**Post date:** [October 6, 2021, 9:29pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/13 "2021-10-06T21:29:52Z")

</div>

In mathematics you can have vector or scalar valued functions of vectors, or any kind of mapping you want to define. You explain what the symbols mean and usually have a typographic convention for them that varies from paper to paper and textbook to textbook. People use boldface, italics, calligraphic fonts, etc. One thing you don’t need is funny dots or stars.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [October 6, 2021, 9:44pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/14 "2021-10-06T21:44:10Z")

</div>

> [@mthelm85](#):
>
> I hereby declare that \star ⋆\star indicates the element-wise application of a function to a vector.

I would just use Julia’s notation of f.(x), honestly. There’s nothing “standard” here, but that notation doesn’t seem to be used for anything else commonplace, it matches the code, and it is inspired by the longstanding “dot” notation in Matlab for element-wise operators, with `.` standing for “pointwise” or something like that.

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [October 7, 2021, 4:28pm UTC](https://discourse.julialang.org/t/broadcasting-mathematical-notation/69327/15 "2021-10-07T16:28:12Z")

</div>

`f_k(x) = \sigma(b_k + w_kx)`