Sigmoid function

Specific Domains Machine Learning
1

Hello,
I have difficulty with getting the derivate (Gradient ) of sigmoid!
using Flux
sigmoid(x)=1 ./(1 .+exp(.-x))
sigmoid(x)=s
d(s)=x .* (1 .- x)
d([1 2 3])
it causes error or not outputting correct number!
I would appreciate if you guys can help me,
thank you

2 
sigmoid(x)=1 ./(1 .+exp(.-x))
sigmoid(x)=s

The second line is replacing the first definition.

1 Like
3

Also in the function above, x is not defined, s should be x.

4

I erased it, but its still causing error !

5

I changed s to x but the output is still the same !

6

Now the code is :
Using Flux
Sigmoid(x)= 1 ./(1 .+exp(.-x))
d(x)=x .* (1 .- x)
d([1 2 3])
Wrong output!

7 
julia> using ForwardDiff

julia> sigmoid(x)= @. 1 / (1 + exp(-x))
sigmoid (generic function with 1 method)

julia> d(x) = @. x * (1 - x)
d (generic function with 1 method)

julia> dsigmoid(x) = d(sigmoid(x))
dsigmoid (generic function with 1 method)

julia> ForwardDiff.derivative(sigmoid, 0.57)
0.23074478122261177

julia> dsigmoid(0.57)
0.23074478122261183

Note that I think it’s better not to add dots in the definitions of sigmoid and d, but to instead add them them where you’re calling them. But I have very little experience with Flux, so maybe things are different there.

8

I tried the following

using Flux
x = [-2:2...]
## Defining sigmoid function
Sigmoid(x) = 1 / (1 + exp(-x))
Sigmoid.(x)
# Getting the gradient
dsigmoid(x) = Flux.gradient(Sigmoid, x)[1]
dsigmoid.(x)

Also sigmoid is already exported by Flux:

using Flux
sigmoid(0) #0.5
9

It worked!!
I’m very grateful for your greatest help rafael !

1 Like