What’s the proper notation for something like this?
x = [0.09, 0.2, 0.05 ,0.01]
w = [0.81 0.92 0.86 0.32; 0.42 0.95 0.96 0.53]
b = [0.57, 0.52]
f(x) = σ.(b + w * x)
f(x) = \sigma.(b + wx)
Is there a common notation that I can just plug into the spot where the dot is or do I need something entirely different?
Not entirely sure I understand the question, but, presumably, the thing that goes where the dot is, is, well, a dot.
I mean, doesn’t it already work? You should probably add a dot to the plus sign as well, though.
In my experience, conventions are not always consistent across fields. This might be helpful. If in doubt, define the notation for your readers.
Oh, you were asking how to denote elementwise application in maths notation? I didn’t get that.
Oh, you were asking how to denote elementwise application in maths notation ? I didn’t get that.
Yes, sorry I wasn’t clear on that. f(x) just applies the logistic/sigmoid function to each element in the 2 x 1 vector that is the result of b + wx. In Julia code, it’s clear to me that I’m applying sigmoid to each element of the vector (because of the dot) but I’m trying to write a small presentation for non-Julians that explains the code and I couldn’t find a standard way to notate that operation…
Since this is just for an internal (work) audience, I’m going to just define my own notation 
I hereby declare that \star indicates the element-wise application of a function to a vector.
f(x) = \sigma \star(b + wx)
because it just looks cool 
I just checked with my fellow mathematicians, and there is no standard notation for that.
Check this post, and take W(x) = b + w*x
Mostly I (and I think people in general) think of broadcasting as a computational convenience.
But it shows up in a suprisingly common bit of calculus (though I might be out of touch with what people do with calculus).
The pullback of y=sum(f, xs; dims) is to broadcast the pullback of f at each element of xs against ȳ.
This broadcasting is exactly what is needed to handle the different dimensionality of of xs and ȳ if the dims argument use used.
See this code
Broadcast multiplication is sometimes called the Hadamard Product.
I don’t think there’s any general, well known mathematical notation for our full broadcasting semantics though
In mathematics you can have vector or scalar valued functions of vectors, or any kind of mapping you want to define. You explain what the symbols mean and usually have a typographic convention for them that varies from paper to paper and textbook to textbook. People use boldface, italics, calligraphic fonts, etc. One thing you don’t need is funny dots or stars.
I would just use Julia’s notation of f.(x), honestly. There’s nothing “standard” here, but that notation doesn’t seem to be used for anything else commonplace, it matches the code, and it is inspired by the longstanding “dot” notation in Matlab for element-wise operators, with . standing for “pointwise” or something like that.
f_k(x) = \sigma(b_k + w_kx)