# Get DCT coefficients of jpeg image

**URL:** <https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640>\
**Category:** Visualization\
**Tags:** question, images\
**Created:** [July 14, 2021, 3:08pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640 "2021-07-14T15:08:24Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![pakk-minidose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pakk-minidose/32/27126_2.png) [@pakk-minidose](https://discourse.julialang.org/u/pakk-minidose)\
**Post date:** [July 14, 2021, 3:08pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/1 "2021-07-14T15:08:24Z")

</div>

Hello,  
I would like to obtain DCT coefficients of a jpeg image.  
The Phil Sallee’s JPEG toolbox in MATLAB returns these as an array (the original website is not active, but some of the code can be downloaded [on Jessica Fridrich’s website](http://dde.binghamton.edu/download/nsf5simulator/) under the DOWNLOAD heading)  
How can I obtain the coefficients using Julia?  
Thank you.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 14, 2021, 4:46pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/2 "2021-07-14T16:46:07Z")

</div>

Glancing at Sallee’s source code, it seems to be a very thin Matlab MEX wrapper around [libjpeg](http://libjpeg.sourceforge.net/), which is a popular C library to access JPEG files.

It should be pretty easy to write a Julia package to call libjpeg, or even just a few lines to extract the information you specifically want. We _already_ have a Yggdrasil package ([JpegTurbo\_jll](https://juliahub.com/ui/Packages/JpegTurbo_jll/EC0Oy/2.1.0+0?t=0)) to provide a compiled libjpeg binary.

In the meantime you could try one of the Python libjpeg-based libraries, which can be called from Julia via PyCall.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 14, 2021, 7:51pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/3 "2021-07-14T19:51:48Z")

</div>

> [@stevengj](#):
>
> It should be pretty easy to write a Julia package to call libjpeg

I took a look at this, and it’s not such a simple task for beginners because of the complexity of the libjpeg API. However, I put together a start at:

> **[GitHub - stevengj/JpegTurbo.jl: Julia interface to libjpeg-turbo](https://github.com/stevengj/JpegTurbo.jl)**
>
> Julia interface to libjpeg-turbo. Contribute to stevengj/JpegTurbo.jl development by creating an account on GitHub.

which at least deals with the initial task of wrapping the `jpeglib.h` header file and mirroring all of the data structures (with help from Clang.jl). As explained in the README, it is now possible to use the library to read a JPEG header and call the low-level API functions directly.

If someone is interested in building on this to put together a higher-level Julia API, let me know.

_Update_: looks like this package by @ianshmean also provides low-level wrappers for libjpeg, and is a bit farther along: [GitHub - IanButterworth/ImageIODevelopment.jl: Developing better IO for images in Julia via libpng, libjpeg, libtiff etc.](https://github.com/IanButterworth/ImageIODevelopment.jl)

---

<div class="post-metadata">

**Author:** ![ianshmean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ianshmean/32/216042_2.png) [@ianshmean](https://discourse.julialang.org/u/ianshmean)\
**Post date:** [July 14, 2021, 8:21pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/4 "2021-07-14T20:21:03Z")

</div>

Indeed. It’d be great to make a JPEGFiles.jl for inclusion in [ImageIO.jl](https://github.com/JuliaIO/ImageIO.jl). I won’t be able to push that forward, but very happy to review.

As well as the ImageIODevelopment.jl repo, I’d recommend looking at [PNGFiles.jl](https://github.com/JuliaIO/PNGFiles.jl) which is a pretty finished product thanks in large part to @drvi

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 14, 2021, 9:21pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/5 "2021-07-14T21:21:14Z")

</div>

> [@pakk-minidose](#):
>
> The Phil Sallee’s JPEG toolbox in MATLAB returns these as an array (the original website is not active, but some of the code can be downloaded [on Jessica Fridrich’s website](http://dde.binghamton.edu/download/nsf5simulator/) under the DOWNLOAD heading)

I took a crack at translating this code into Julia using the JpegTurbo.jl package I linked above:

```julia
import JpegTurbo: LibJpeg
import JpegTurbo.LibJpeg: DCTSIZE

function jpeg_read_coefficients(io::IO)
    cinfo = LibJpeg.jpeg_decompress_struct()
    jerr = Ref{LibJpeg.jpeg_error_mgr}()
    cinfo.err = LibJpeg.jpeg_std_error(jerr)
    LibJpeg.jpeg_create_decompress(cinfo)
    io_file = Libc.FILE(io)
    LibJpeg.jpeg_stdio_src(cinfo, io_file)
    LibJpeg.jpeg_read_header(cinfo, true)
    
    coef_arrays = LibJpeg.jpeg_read_coefficients(cinfo)
    access_virt_barray = unsafe_load(cinfo.mem).access_virt_barray
    components = Matrix{LibJpeg.JCOEF}[]
    
    for ci = 1:cinfo.num_components
        comp_info = unsafe_load(cinfo.comp_info, ci)
        c_height = comp_info.height_in_blocks * DCTSIZE
        c_width = comp_info.width_in_blocks * DCTSIZE
        m = Matrix{LibJpeg.JCOEF}(undef, c_height, c_width)
        
        for blk_y = 1:comp_info.height_in_blocks
            buffer = unsafe_load(ccall(access_virt_barray, LibJpeg.JBLOCKARRAY, 
                (Ref{LibJpeg.jpeg_decompress_struct},Ptr{Cvoid},LibJpeg.JDIMENSION,LibJpeg.JDIMENSION,LibJpeg.boolean),
                cinfo, unsafe_load(coef_arrays,ci), blk_y-1, 1, false))
            for blk_x = 1:comp_info.width_in_blocks
                bufptr = unsafe_load(buffer, blk_x)
                for i=1:DCTSIZE, j=1:DCTSIZE
                    m[i + DCTSIZE*(blk_y-1), j + DCTSIZE*(blk_x-1)] =
                        bufptr[(i-1)*DCTSIZE + j]
                end
            end
        end
        
        push!(components, m)
    end
    
    LibJpeg.jpeg_destroy_decompress(cinfo)
    close(io_file)
    return components
end

jpeg_read_coefficients(filename::AbstractString) =
    open(jpeg_read_coefficients, filename, "r")

```

It runs, in the sense that `jpeg_read_coefficients(filename)` returns an array of matrices of DCT coefficients (one per component). I’m not sure if it’s correct, since I don’t know what the right output is — it’s possible there is a typo, but at least the above code should be a good start.

_Update_: fixed to explicitly `close` the file as suggested by @pakk-minidose below

---

<div class="post-metadata">

**Author:** ![kimikage](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kimikage/32/14534_2.png) [@kimikage](https://discourse.julialang.org/u/kimikage)\
**Post date:** [July 14, 2021, 11:19pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/6 "2021-07-14T23:19:41Z")

</div>

Incidentally, about 5 years ago, I wrote a JPEG (baseline) decoder as an exercise in learning Julia.  
The original code doesn’t work with Julia v1 and the latest packages, so I’m trying to modernize it little by little, but I’ve been too busy lately. 😭

> **[GitHub - kimikage/LowLevelJPEG.jl: a library for low-level manipulation of...](https://github.com/kimikage/LowLevelJPEG.jl)**
>
> a library for low-level manipulation of JPEG images - GitHub - kimikage/LowLevelJPEG.jl: a library for low-level manipulation of JPEG images

What I want to say is that obtaining DCT coefficients in pure Julia is not technically so difficult. 😄

---

<div class="post-metadata">

**Author:** ![pakk-minidose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pakk-minidose/32/27126_2.png) [@pakk-minidose](https://discourse.julialang.org/u/pakk-minidose)\
**Post date:** [July 15, 2021, 10:57am UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/7 "2021-07-15T10:57:06Z")

</div>

Thank you everybody for your great replies!  
However, when trying out @stevengj 's code, I run into the following error:

```julia
JPEG parameter struct mismatch: library thinks size is 600, caller expects 632

```

The test code was run using Julia 1.6.2 and the code was following:

```julia
using Pkg; Pkg.activate(temp=true) #create empty environment to isolate the issue
Pkg.add(url="https://github.com/stevengj/JpegTurbo.jl#main")
Pkg.add("TestImages")

#copy-paste from Discourse
import JpegTurbo: LibJpeg
import JpegTurbo.LibJpeg: DCTSIZE

function jpeg_read_coefficients(io::IO)
    cinfo = LibJpeg.jpeg_decompress_struct()
    jerr = Ref{LibJpeg.jpeg_error_mgr}()
    cinfo.err = LibJpeg.jpeg_std_error(jerr)
    LibJpeg.jpeg_create_decompress(cinfo)
    LibJpeg.jpeg_stdio_src(cinfo, Libc.FILE(io))
    LibJpeg.jpeg_read_header(cinfo, true)

    coef_arrays = LibJpeg.jpeg_read_coefficients(cinfo)
    access_virt_barray = unsafe_load(cinfo.mem).access_virt_barray
    components = Matrix{LibJpeg.JCOEF}[]

    for ci = 1:cinfo.num_components
        comp_info = unsafe_load(cinfo.comp_info, ci)
        c_height = comp_info.height_in_blocks * DCTSIZE
        c_width = comp_info.width_in_blocks * DCTSIZE
        m = Matrix{LibJpeg.JCOEF}(undef, c_height, c_width)

        for blk_y = 1:comp_info.height_in_blocks
            buffer = unsafe_load(ccall(access_virt_barray, LibJpeg.JBLOCKARRAY,
                (Ref{LibJpeg.jpeg_decompress_struct},Ptr{Cvoid},LibJpeg.JDIMENSION,LibJpeg.JDIMENSION,LibJpeg.boolean),
                cinfo, unsafe_load(coef_arrays,ci), blk_y-1, 1, false))
            for blk_x = 1:comp_info.width_in_blocks
                bufptr = unsafe_load(buffer, blk_x)
                for i=1:DCTSIZE, j=1:DCTSIZE
                    m[i + DCTSIZE*(blk_y-1), j + DCTSIZE*(blk_x-1)] =
                        bufptr[(i-1)*DCTSIZE + j]
                end
            end
        end

        push!(components, m)
    end

    LibJpeg.jpeg_destroy_decompress(cinfo)
    return components
end

jpeg_read_coefficients(filename::AbstractString) =
    open(jpeg_read_coefficients, filename, "r")

#now the test itself
using TestImages
fname = testimage("earth_apollo17", download_only=true)
jpeg_read_coefficients(fname)

```

Info about my Julia installation:

```julia
julia> versioninfo()
Julia Version 1.6.2
Commit 1b93d53fc4 (2021-07-14 15:36 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

```

Thank you for your help.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 15, 2021, 12:27pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/8 "2021-07-15T12:27:05Z")

</div>

> [@pakk-minidose](#):
>
> `JPEG parameter struct mismatch: library thinks size is 600, caller expects 632`

Probably some of the libjpeg types are different under Windows (your machine) vs Mac (my machine). Can you run

```julia
    using Clang, JpegTurbo_jll

    jconfig_h = joinpath(JpegTurbo_jll.artifact_dir, "include", "jconfig.h")
    jmorecfg_h = joinpath(JpegTurbo_jll.artifact_dir, "include", "jmorecfg.h")
    jpeglib_h = joinpath(JpegTurbo_jll.artifact_dir, "include", "jpeglib.h")
    wc = init(; headers = [jconfig_h, jmorecfg_h, jpeglib_h],
            output_file = joinpath(@ __DIR__ , "libjpeg_out.jl"),
            common_file = joinpath(@ __DIR__ , "libjpeg_common.jl"),
            header_wrapped = (root, current)->root == current,
            header_library = x->"libjpeg"
            )
    run(wc)

```

on your machine and compare the `libjpeg_common.jl` file (with `diff` or similar) to the Mac output (at [this gist](https://gist.github.com/stevengj/a2faa46f71bef91b85fa839712f5fcb9)).

---

<div class="post-metadata">

**Author:** ![pakk-minidose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pakk-minidose/32/27126_2.png) [@pakk-minidose](https://discourse.julialang.org/u/pakk-minidose)\
**Post date:** [July 15, 2021, 2:08pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/9 "2021-07-15T14:08:11Z")

</div>

Here is the diff. I below is a screenshot of the output as it is easier to read with its color coding

```plaintext
PS C:\tmp> git diff --no-index .\libjpeg_common_win.jl .\libjpeg_common_mac.jl
warning: LF will be replaced by CRLF in .\libjpeg_common_win.jl.
The file will have its original line endings in your working directory
warning: LF will be replaced by CRLF in .\libjpeg_common_mac.jl.
The file will have its original line endings in your working directory
diff --git "a/.\\libjpeg_common_win.jl" "b/.\\libjpeg_common_mac.jl"
index 6a645da..9452e09 100644
--- "a/.\\libjpeg_common_win.jl"
+++ "b/.\\libjpeg_common_mac.jl"
@@ -4,10 +4,17 @@
 const JPEG_LIB_VERSION = 62
 const LIBJPEG_TURBO_VERSION = ".1."
 const LIBJPEG_TURBO_VERSION_NUMBER = 2001000
+const C_ARITH_CODING_SUPPORTED = 1
+const D_ARITH_CODING_SUPPORTED = 1
+const MEM_SRCDST_SUPPORTED = 1
+const WITH_SIMD = 1
 const BITS_IN_JSAMPLE = 8
-const boolean = Cuchar
-const INT16 = Int16
-const INT32 = Cint
+const HAVE_LOCALE_H = 1
+const HAVE_STDDEF_H = 1
+const HAVE_STDLIB_H = 1
+const NEED_SYS_TYPES_H = 1
+const HAVE_UNSIGNED_CHAR = 1
+const HAVE_UNSIGNED_SHORT = 1
 const MAX_COMPONENTS = 10

 # Skipping MacroDefinition: GETJOCTET ( value ) ( value )
@@ -26,7 +33,10 @@ const JCOEF = Int16
 const JOCTET = Cuchar
 const UINT8 = Cuchar
 const UINT16 = UInt32
+const INT16 = Int16
+const INT32 = Clong
 const JDIMENSION = UInt32
+const boolean = Cint
 const DCTSIZE = 8
 const DCTSIZE2 = 64
 const NUM_QUANT_TBLS = 4

```

 ![obrazek](https://global.discourse-cdn.com/julialang/original/3X/b/3/b33c828fcd8ffe071ae8f41e89c5c3000325adda.png)  
Thank you.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 15, 2021, 2:41pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/10 "2021-07-15T14:41:06Z")

</div>

> [@pakk-minidose](#):
>
> Here is the diff. I below is a screenshot of the output as it is easier to read with its color coding

It looks like the culprit is the `boolean` type, which is apparently `unsigned char` on Windows and `int` on Mac. I’ve pushed an update that should fix this.

---

<div class="post-metadata">

**Author:** ![pakk-minidose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pakk-minidose/32/27126_2.png) [@pakk-minidose](https://discourse.julialang.org/u/pakk-minidose)\
**Post date:** [July 19, 2021, 9:57am UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/11 "2021-07-19T09:57:05Z")

</div>

Great! It now works on Windows. I’ve done a very coarse comparison with the MATLAB output and it seems to do the same.  
Thank you for your help!

---

<div class="post-metadata">

**Author:** ![pakk-minidose](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pakk-minidose/32/27126_2.png) [@pakk-minidose](https://discourse.julialang.org/u/pakk-minidose)\
**Post date:** [July 21, 2021, 2:05pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/12 "2021-07-21T14:05:59Z")

</div>

Just adding a small bugfix.  
It seems that the Julia Libc `FILE` does not release the file descriptor even though it is not reachable anymore, see [this comment in the source code](https://github.com/JuliaLang/julia/blob/1b93d53fc4bb59350ada898038ed4de2994cce33/base/libc.jl#L73). I found out when my Julia instance ran out of file descriptors (Linux limits number of file descriptors per process to 1024) and the code above kept throwing `SystemError: fdopen: Bad file descriptor` in the `Libc.FILE(io)` call.  
Here is a version which closes the file descriptor for anyone who would like to run the code

```julia
function jpeg_read_coefficients(io::IO)
    cinfo = LibJpeg.jpeg_decompress_struct()
    jerr = Ref{LibJpeg.jpeg_error_mgr}()
    cinfo.err = LibJpeg.jpeg_std_error(jerr)
    LibJpeg.jpeg_create_decompress(cinfo)
    fle = Libc.FILE(io)
    LibJpeg.jpeg_stdio_src(cinfo, fle)
    LibJpeg.jpeg_read_header(cinfo, true)

    coef_arrays = LibJpeg.jpeg_read_coefficients(cinfo)
    access_virt_barray = unsafe_load(cinfo.mem).access_virt_barray
    components = Matrix{LibJpeg.JCOEF}[]

    for ci = 1:cinfo.num_components
        comp_info = unsafe_load(cinfo.comp_info, ci)
        c_height = comp_info.height_in_blocks * DCTSIZE
        c_width = comp_info.width_in_blocks * DCTSIZE
        m = Matrix{LibJpeg.JCOEF}(undef, c_height, c_width)

        for blk_y = 1:comp_info.height_in_blocks
            buffer = unsafe_load(ccall(access_virt_barray, LibJpeg.JBLOCKARRAY,
                (Ref{LibJpeg.jpeg_decompress_struct},Ptr{Cvoid},LibJpeg.JDIMENSION,LibJpeg.JDIMENSION,LibJpeg.boolean),
                cinfo, unsafe_load(coef_arrays,ci), blk_y-1, 1, false))
            for blk_x = 1:comp_info.width_in_blocks
                bufptr = unsafe_load(buffer, blk_x)
                for i=1:DCTSIZE, j=1:DCTSIZE
                    m[i + DCTSIZE*(blk_y-1), j + DCTSIZE*(blk_x-1)] =
                        bufptr[(i-1)*DCTSIZE + j]
                end
            end
        end

        push!(components, m)
    end

    LibJpeg.jpeg_destroy_decompress(cinfo)
    close(fle)
    return components
end

jpeg_read_coefficients(filename::AbstractString) =
    open(jpeg_read_coefficients, filename, "r")

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 21, 2021, 3:42pm UTC](https://discourse.julialang.org/t/get-dct-coefficients-of-jpeg-image/64640/13 "2021-07-21T15:42:54Z")

</div>

5 posts were split to a new topic: [Corrections to other posts: best practices?](https://discourse.julialang.org/t/corrections-to-other-posts-best-practices/65044)