# 1D GAN with Flux

**URL:** <https://discourse.julialang.org/t/1d-gan-with-flux/54179>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [January 29, 2021, 12:52pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179 "2021-01-29T12:52:22Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![davibarreira](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davibarreira/32/20270_2.png) [@davibarreira](https://discourse.julialang.org/u/davibarreira)\
**Post date:** [January 29, 2021, 12:52pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/1 "2021-01-29T12:52:22Z")

</div>

Hey everyone. So I’ve been trying to use Flux for Generative Adversarial Networks (GAN). And unfortunately, it’s not working properly. At the moment, I’m trying to replicate a 1D GAN example (this one here [1D GAN with Keras](https://machinelearningmastery.com/how-to-develop-a-generative-adversarial-network-for-a-1-dimensional-function-from-scratch-in-keras/)). But the results are just not good. First, the discriminator takes a lot to properly learn to discern the data, and the generator is even worse.

I was wondering if anyone here has implemented GANs on Flux, and perhaps can show how to implement this 1D example. I’ve based my code on the DCGAN example from the model\_zoo.

---

<div class="post-metadata">

**Author:** ![darsnack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/darsnack/32/10144_2.png) [@darsnack](https://discourse.julialang.org/u/darsnack)\
**Post date:** [January 29, 2021, 4:29pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/2 "2021-01-29T16:29:14Z")

</div>

Can you post the full Julia code for us to compare?

---

<div class="post-metadata">

**Author:** ![davibarreira](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davibarreira/32/20270_2.png) [@davibarreira](https://discourse.julialang.org/u/davibarreira)\
**Post date:** [January 29, 2021, 10:01pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/3 "2021-01-29T22:01:11Z")

</div>

Sure, so here is my code trying to replicate this 1-D GAN:

```julia
function generate_real_data(n)
    x1 = rand(1,n) .- 0.5
    x2 = x1 .* x1
    return vcat(x1,x2)
end

function D()
    return Chain(
            Dense(2, 25,relu),
            Dense(25,1,x->σ.(x))
            )
end

function G(latent_dim::Int)
    return Chain(
            Dense(latent_dim, 15,relu),
            Dense(15,2)
            )
end
loss_D(x, y) = sum(Flux.Losses.logitbinarycrossentropy(dscr(x), y))

function trainDiscriminator!(dscr,gen,train_size)
    real = generate_real_data(train_size)
    fake = gen(rand(5,train_size))

    X = hcat(real,fake)
    Y = vcat(ones(train_size),zeros(train_size))
    data = Flux.Data.DataLoader(X, Y', batchsize=100,shuffle=true);
    for d in data
        gs = gradient(Flux.params(dscr)) do
            l = loss_D(d...)
        end
        Flux.update!(opt, Flux.params(dscr), gs)
    end
end

loss_G(x) = sum(Flux.Losses.logitbinarycrossentropy(dscr(x),1))

function trainGenerator!(gen,dscr,train_size)
    fake_generated = gen(rand(5,train_size))
    data = Flux.Data.DataLoader(fake_generated, batchsize=100,shuffle=true);
    for d in data
        gs = gradient(Flux.params(gen)) do
            l = loss_G(d)
        end
        Flux.update!(opt, Flux.params(gen), gs)
    end
    fake_generated = gen(rand(5,train_size))
end

gen = G(5)
dscr = D()
opt = ADAM()
train_size = 2000

epochs = 10000
for e in 1:epochs
    trainDiscriminator!(dscr,gen,train_size)
    trainGenerator!(gen,dscr,train_size)
    if e%1000 == 0
        real = generate_real_data(train_size)
        fake = gen(rand(5,train_size))
        @show mean(dscr(real)),mean(dscr(fake))
    end
end

real = generate_real_data(train_size)
fake = gen(rand(5,train_size))
scatter(real[1,1:100],real[2,1:100])
scatter!(fake[1,1:100],fake[2,1:100])

```

---

<div class="post-metadata">

**Author:** ![jondeuce](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jondeuce/32/16378_2.png) [@jondeuce](https://discourse.julialang.org/u/jondeuce)\
**Post date:** [January 30, 2021, 6:43am UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/4 "2021-01-30T06:43:42Z")

</div>

Your generator loss is not actually differentiating through the generator; you are sampling the noise distribution, passing those through the generator, and then passing precomputed generator samples to the data loader. You need to pass noise distribution samples to the data loader and call the generator on those noise instances _within the loss function_. This allows the generator to be differentiated through.

Also, your `loss_D` and `loss_G` functions rely on `dscr` implicitly; it is probably best to explicitly pass the models as arguments (same goes for `gen`):

```julia
loss_D(dscr, x, y) = sum(Flux.Losses.logitbinarycrossentropy(dscr(x), y))

loss_G(dscr, gen, z) = sum(Flux.Losses.logitbinarycrossentropy(dscr(gen(z)), 1))

```

---

<div class="post-metadata">

**Author:** ![davibarreira](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davibarreira/32/20270_2.png) [@davibarreira](https://discourse.julialang.org/u/davibarreira)\
**Post date:** [January 30, 2021, 12:45pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/5 "2021-01-30T12:45:33Z")

</div>

Thanks a lot for the reply. Yet, the generator still does not replicate like the example [here](https://machinelearningmastery.com/how-to-develop-a-generative-adversarial-network-for-a-1-dimensional-function-from-scratch-in-keras/). So, here is my updated code:

```julia
loss_D(x, y, dscr) = sum(Flux.Losses.logitbinarycrossentropy(dscr(x), y))

function trainDiscriminator!(dscr,gen,train_size)
    real = generate_real_data(train_size)
    fake = gen(rand(5,train_size))

    X = hcat(real,fake)
    Y = vcat(ones(train_size),zeros(train_size))
    data = Flux.Data.DataLoader(X, Y', batchsize=128,shuffle=true);
    for d in data
        gs = gradient(Flux.params(dscr)) do
            l = loss_D(d...,dscr)
        end
        Flux.update!(opt, Flux.params(dscr), gs)
    end
end

loss_G(z,gen,dscr) = sum(Flux.Losses.logitbinarycrossentropy(dscr(gen(z)),1))

function trainGenerator!(gen,dscr,train_size)
    noise = rand(5,train_size)
    data = Flux.Data.DataLoader(noise, batchsize=128,shuffle=true);
    for d in data
        gs = gradient(Flux.params(gen)) do
            l = loss_G(d,gen,dscr)
        end
        Flux.update!(opt, Flux.params(gen), gs)
    end
    fake_generated = gen(rand(5,train_size))
end

```

---

<div class="post-metadata">

**Author:** ![davibarreira](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davibarreira/32/20270_2.png) [@davibarreira](https://discourse.julialang.org/u/davibarreira)\
**Post date:** [January 30, 2021, 1:06pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/6 "2021-01-30T13:06:11Z")

</div>

Finally I got it to work! So the problem was in this sigmoid in the Discriminator and the fact that I was using the logit in the loss function. I removed the sigmoid, and now is all good!

---

<div class="post-metadata">

**Author:** ![sanjay\_chawla](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@sanjay\_chawla](https://discourse.julialang.org/u/sanjay_chawla)\
**Post date:** [April 11, 2021, 4:31pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/7 "2021-04-11T16:31:34Z")

</div>

Hello…can you please post the full code. Thanks.

---

<div class="post-metadata">

**Author:** ![jfcal](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jfcal](https://discourse.julialang.org/u/jfcal)\
**Post date:** [August 13, 2021, 9:36pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/8 "2021-08-13T21:36:32Z")

</div>

plus 1 for the code!

---

<div class="post-metadata">

**Author:** ![logankilpatrick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/logankilpatrick/32/38123_2.png) [@logankilpatrick](https://discourse.julialang.org/u/logankilpatrick)\
**Post date:** [September 18, 2021, 3:40am UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/9 "2021-09-18T03:40:23Z")

</div>

@davibarreira Can you post the full code here, even better yet maybe a quick tutorial on: [https://github.com/FluxML/fluxml.github.io/tree/main/tutorials/\_posts](https://github.com/FluxML/fluxml.github.io/tree/main/tutorials/_posts) ?

---

<div class="post-metadata">

**Author:** ![jfcal](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jfcal](https://discourse.julialang.org/u/jfcal)\
**Post date:** [October 29, 2021, 2:07pm UTC](https://discourse.julialang.org/t/1d-gan-with-flux/54179/10 "2021-10-29T14:07:27Z")

</div>

Please check this code over here that I made, must be at least similar to the pretended

[https://github.com/joofio/ds-curriculum/blob/main/generative/1D-GAN-julia.ipynb](https://github.com/joofio/ds-curriculum/blob/main/generative/1D-GAN-julia.ipynb)