# Image Super Resolution

**URL:** <https://discourse.julialang.org/t/image-super-resolution/48282>\
**Category:** Visualization\
**Tags:** machinevision\
**Created:** [October 13, 2020, 8:22am UTC](https://discourse.julialang.org/t/image-super-resolution/48282 "2020-10-13T08:22:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kungfooman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kungfooman/32/3395_2.png) [@kungfooman](https://discourse.julialang.org/u/kungfooman)\
**Post date:** [October 13, 2020, 8:22am UTC](https://discourse.julialang.org/t/image-super-resolution/48282/1 "2020-10-13T08:22:48Z")

</div>

Hi, is there any Julia project focusing on Image Super Resolution? There are lots of these libraries for e.g. Python or Mathematica, but found not one for Julia:

> **[A Model for Super-Resolution: New in Wolfram Language 12](https://www.wolfram.com/language/12/machine-learning-for-images/a-model-for-super-resolution.html?product=mathematica)**

> **[Image Super Resolution using ESRGAN  |  TensorFlow Hub](https://www.tensorflow.org/hub/tutorials/image_enhancing?hl=en)**

---

<div class="post-metadata">

**Author:** ![Zach\_Christensen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zach_christensen/32/7220_2.png) [@Zach\_Christensen](https://discourse.julialang.org/u/Zach_Christensen)\
**Post date:** [October 15, 2020, 12:03am UTC](https://discourse.julialang.org/t/image-super-resolution/48282/2 "2020-10-15T00:03:21Z")

</div>

I’m assuming you’re referring to super resolution specifically in terms of neural networks.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 15, 2020, 10:39am UTC](https://discourse.julialang.org/t/image-super-resolution/48282/3 "2020-10-15T10:39:21Z")

</div>

What you post is likely outdated (ESRGAN from 2018, vs from 2020 I link to below, see Tencent link for impressive pictures, and the problems they solved, “[old] methods always fail in real-world image super-resolution”). If you want to reimplement something in Julia, please use state-of-the-art, such as I link to below (and you likely do not have to, can just call Python code?).

First what I see is available in Julia (I didn’t look closely, it’s probably not based on neural-network):

The slides here:  
[https://web.eecs.umich.edu/~fessler/papers/files/talk/19/ima-julia/](https://web.eecs.umich.edu/~fessler/papers/files/talk/19/ima-julia/)  
Slide 15: Super-resolution example

> using MIRT: jim, ellipse\_im

> **[GitHub - JeffFessler/MIRT.jl: MIRT: Michigan Image Reconstruction Toolbox...](https://github.com/JeffFessler/MIRT.jl)**
>
> MIRT: Michigan Image Reconstruction Toolbox (Julia version) - GitHub - JeffFessler/MIRT.jl: MIRT: Michigan Image Reconstruction Toolbox (Julia version)

See at: [https://web.eecs.umich.edu/~fessler/papers/files/talk/19/ima-julia/](https://web.eecs.umich.edu/~fessler/papers/files/talk/19/ima-julia/)

[https://ece.engin.umich.edu/event/seminar-by-jeff-fessler](https://ece.engin.umich.edu/event/seminar-by-jeff-fessler)

[https://fluxml.ai/2019/05/31/JSoC-Cohort.html](https://fluxml.ai/2019/05/31/JSoC-Cohort.html)

> ## Shreyas Kowshik: _Addition Of Baseline Models To Flux.jl_
> 
> The current state of the art algorithms in terms of reinforcement learning and generative models are not yet available in [Flux](https://github.com/FluxML). This project aims to add the following models with explicit documentation in the form of [blog](https://shreyas-kowshik.github.io/) and code :
> 
> - […]
> - Super Resolution GAN

Interesting links, first what I knew about, then something I dug up now:

> **[GitHub - Tencent/Real-SR: Real-World Super-Resolution via Kernel Estimation...](https://github.com/Tencent/Real-SR)**
>
> Real-World Super-Resolution via Kernel Estimation and Noise Injection - GitHub - Tencent/Real-SR: Real-World Super-Resolution via Kernel Estimation and Noise Injection

> In addition, our method is the winner of NTIRE 2020 Challenge on both tracks of Real-World Super-Resolution, which significantly outperforms other competitors by large margins.

This one also scored high:

> **[GitHub - sunwj/CAR: Content adaptive resampler for image downscaling](https://github.com/sunwj/CAR)**
>
> Content adaptive resampler for image downscaling. Contribute to sunwj/CAR development by creating an account on GitHub.

And something specific to faces might be useful:

> **[GitHub - ieee820/FSRNet: Demo code for our CVPR'18 paper "FSRNet: End-to-End...](https://github.com/ieee820/FSRNet)**
>
> Demo code for our CVPR'18 paper "FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors" (SPOTLIGHT Presentation) - GitHub - ieee820/FSRNet: Demo code for our CVPR'1...

> **[DeepSUM: Deep neural network for Super-resolution of Unregistered...](https://arxiv.org/abs/1907.06490)**
>
> Recently, convolutional neural networks (CNN) have been successfully applied to many remote sensing problems. However, deep learning techniques for multi-image super-resolution from multitemporal unregistered imagery have received little attention so...

Didn’t know about this, it’s pretty neat:

> **[Pixel shift](https://en.wikipedia.org/wiki/Pixel_shift)**
>
> Pixel shift is a method in digital cameras for producing a super resolution image. The method works by taking several images, after each such capture moving ("shifting") the sensor to a new position. In digital colour cameras that employ pixel shift, this avoids a major limitation inherent in using Bayer pattern for obtaining colour, and instead produces an image with increased colour resolution and, assuming a static subject or additional computational steps, an image free of colour moiré. Takin...

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 15, 2020, 11:32am UTC](https://discourse.julialang.org/t/image-super-resolution/48282/4 "2020-10-15T11:32:15Z")

</div>

Some additional links I had.

See there more amazing upscaling pictures, and source code (for Tencent link above), and executables for Linux Windows, macOS (that you could use independently, or from Julia or the underlying C++ library, even on Android and iOS too):

[https://github.com/nihui/realsr-ncnn-vulkan](https://github.com/nihui/realsr-ncnn-vulkan)

> Upscale 4x with ImageMagick Lanczo4 Filter  
> [vs]  
> Upscale 4x with realsr model=DF2K scale=4 tta=1  
> realsr-ncnn-vulkan.exe -i origin.jpg -o output.png -s 4 -x -m models-DF2K

“-resize 400%”, i.e. 4x may be standard terminology, or not, for 16x larger.

FYI: I also see the term “Extreme Super-Resolution” used, but I believe that’s the same magnification as above, as I do not really believe 16x16=256x magnification:

> **[NTIRE 2020 Challenge on Perceptual Extreme Super-Resolution: Methods and Results](https://ieeexplore.ieee.org/document/9150605)**
>
> This paper reviews the NTIRE 2020 challenge on perceptual extreme super-resolution with focus on proposed solutions and results. The challenge task was to super-resolve an input image with a magnification factor ×16 based on a set of prior examples...

> **Abstract:**
> 
> This paper reviews the NTIRE 2020 challenge on perceptual extreme super-resolution with focus on proposed solutions and results. The challenge task was to super-resolve an input image with a magnification factor ×16 based on a set of prior examples of low and corresponding high resolution images.

See also here, where you can compare 7 methods across several pictures:

## Blind Super-Resolution Methods comparison

[http://www.wisdom.weizmann.ac.il/~vision/kernelgan/resources/more\_examples.html](http://www.wisdom.weizmann.ac.il/~vision/kernelgan/resources/more_examples.html)