# Flux bilinear upsampling

**URL:** <https://discourse.julialang.org/t/flux-bilinear-upsampling/37717>\
**Category:** Machine Learning\
**Created:** [April 16, 2020, 9:34pm UTC](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717 "2020-04-16T21:34:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kevinczimmerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevinczimmerman/32/13040_2.png) [@kevinczimmerman](https://discourse.julialang.org/u/kevinczimmerman)\
**Post date:** [April 16, 2020, 9:34pm UTC](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717/1 "2020-04-16T21:34:47Z")

</div>

What is the best way to use Flux to create a bilinear upsampling layer? Something analogous to [tensorflow bilinear upsample](https://www.tensorflow.org/api_docs/python/tf/keras/layers/UpSampling2D).

I appreciate your insight in advance.

---

<div class="post-metadata">

**Author:** ![kevinczimmerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevinczimmerman/32/13040_2.png) [@kevinczimmerman](https://discourse.julialang.org/u/kevinczimmerman)\
**Post date:** [April 22, 2020, 12:09am UTC](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717/2 "2020-04-22T00:09:25Z")

</div>

Decided to [go for it](https://github.com/FluxML/Flux.jl/pull/1136).

---

<div class="post-metadata">

**Author:** ![LeoKoo](https://avatars.discourse-cdn.com/v4/letter/l/ccd318/32.png) [@LeoKoo](https://discourse.julialang.org/u/LeoKoo)\
**Post date:** [May 14, 2020, 9:28am UTC](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717/3 "2020-05-14T09:28:51Z")

</div>

The implementation effort has moved to [this pull request](https://github.com/FluxML/Flux.jl/pull/1180).

I got the forward part working for both CPU and GPU, but `gradient` doesn’t work in the GPU case (it does in the CPU case). It seems hard to debug because Juno’s debugger crashes when I try to get into the `gradient` call. Who here has more experience getting gradients through with CuArrays?

```julia
x = Float32.([1 2; 3 4])[:,:,:,:]
x_c = CuArray(x)

c = BilinearUpsample2d((2,2))
c_c = gpu(c) #don't think this does anything, since there are no arrays stored

o = c(x)
o_c = c(x_c)

g = gradient(x -> sum(c(x)), x)[1]
g_c = gradient(x -> sum(c(x)), x_c)[1]
@Juno.run gradient(x -> sum(c(x)), x_c) #crashes on my setup

```

The error it gives is “unrecognized isdefined node $(QuoteNode(Float32))”, can’t find anything on it

My setup is Julia 1.4.1, Flux 0.10.5, Atom 1.46.0

Any ideas?

---

<div class="post-metadata">

**Author:** ![maxfreu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maxfreu/32/17468_2.png) [@maxfreu](https://discourse.julialang.org/u/maxfreu)\
**Post date:** [July 3, 2020, 12:21pm UTC](https://discourse.julialang.org/t/flux-bilinear-upsampling/37717/4 "2020-07-03T12:21:49Z")

</div>

Hi, I cannot directly help with your problem, but maybe we can find another solution in separating CPU and GPU code. I have translated pytorch’s implementation of bilinear upsampling to julia and it seems to work more or less, even with fractional upsampling. I just dont have experience with zygote and adjoints. You can have a look at the implementation [here](https://gist.github.com/maxfreu/815c8e0c8304bab39d7f78033289cece) and try it out with the following code. The forward pass looks good, but I havent checked the backward pass.

```julia
using FileIO
using ImageView # ZZZzzzz
using Colors
using BenchmarkTools
f = download("https://upload.wikimedia.org/wikipedia/en/e/ed/Nyan_cat_250px_frame.PNG")
nyan = load(f)

imshow(nyan)

nyan_nchw = reshape(reinterpret(UInt8, nyan),1,3,250,250)
nyan_whcn = permutedims(nyan_nchw, [3,4,2,1]) .|> Float32
nyan_gpu = CuArray(nyan_whcn)

nyan_large = upsample_bilinear(nyan_gpu, pi, pi)

nyan_large_cpu = Array(nyan_large)[:,:,:,1]
nyan_colored = view(reinterpret(RGB{Float32}, permutedims(nyan_large_cpu, [3,1,2])),1,:,:) # what a mess

imshow(nyan_colored/255)

# or plain arrays:
n = 64
s = 4
checkerboard = zeros(Float32, n, n)
for x in 1:2s:n-s
    for y in 1:2s:n-s
        checkerboard[y:y+s-1, x:x+s-1] .= 1
    end
end
imshow(checkerboard)

checkerboard_gpu = CuArray(reshape(checkerboard,n,n,1,1))

res = Array(upsample_bilinear(checkerboard_gpu, 2, 2))

imshow(res[:,:,1,1])
imshow(round.(res[:,:,1,1]))

```

Maybe we you can compare this to your results and have a look at the backward pass.

EDIT: I just saw [this](https://discourse.julialang.org/t/ann-announcing-torch-jl/42390), which also looks interesting!