Hello,
I would like to obtain DCT coefficients of a jpeg image.
The Phil Sallee’s JPEG toolbox in MATLAB returns these as an array (the original website is not active, but some of the code can be downloaded on Jessica Fridrich’s website under the DOWNLOAD heading)
How can I obtain the coefficients using Julia?
Thank you.
1 Like
Glancing at Sallee’s source code, it seems to be a very thin Matlab MEX wrapper around libjpeg, which is a popular C library to access JPEG files.
It should be pretty easy to write a Julia package to call libjpeg, or even just a few lines to extract the information you specifically want. We already have a Yggdrasil package (JpegTurbo_jll) to provide a compiled libjpeg binary.
In the meantime you could try one of the Python libjpeg-based libraries, which can be called from Julia via PyCall.
I took a look at this, and it’s not such a simple task for beginners because of the complexity of the libjpeg API. However, I put together a start at:
which at least deals with the initial task of wrapping the jpeglib.h header file and mirroring all of the data structures (with help from Clang.jl). As explained in the README, it is now possible to use the library to read a JPEG header and call the low-level API functions directly.
If someone is interested in building on this to put together a higher-level Julia API, let me know.
Update: looks like this package by @ianshmean also provides low-level wrappers for libjpeg, and is a bit farther along: GitHub - IanButterworth/ImageIODevelopment.jl: Developing better IO for images in Julia via libpng, libjpeg, libtiff etc.
2 Likes
Indeed. It’d be great to make a JPEGFiles.jl for inclusion in ImageIO.jl. I won’t be able to push that forward, but very happy to review.
As well as the ImageIODevelopment.jl repo, I’d recommend looking at PNGFiles.jl which is a pretty finished product thanks in large part to @drvi
2 Likes
I took a crack at translating this code into Julia using the JpegTurbo.jl package I linked above:
import JpegTurbo: LibJpeg
import JpegTurbo.LibJpeg: DCTSIZE
function jpeg_read_coefficients(io::IO)
cinfo = LibJpeg.jpeg_decompress_struct()
jerr = Ref{LibJpeg.jpeg_error_mgr}()
cinfo.err = LibJpeg.jpeg_std_error(jerr)
LibJpeg.jpeg_create_decompress(cinfo)
io_file = Libc.FILE(io)
LibJpeg.jpeg_stdio_src(cinfo, io_file)
LibJpeg.jpeg_read_header(cinfo, true)
coef_arrays = LibJpeg.jpeg_read_coefficients(cinfo)
access_virt_barray = unsafe_load(cinfo.mem).access_virt_barray
components = Matrix{LibJpeg.JCOEF}[]
for ci = 1:cinfo.num_components
comp_info = unsafe_load(cinfo.comp_info, ci)
c_height = comp_info.height_in_blocks * DCTSIZE
c_width = comp_info.width_in_blocks * DCTSIZE
m = Matrix{LibJpeg.JCOEF}(undef, c_height, c_width)
for blk_y = 1:comp_info.height_in_blocks
buffer = unsafe_load(ccall(access_virt_barray, LibJpeg.JBLOCKARRAY,
(Ref{LibJpeg.jpeg_decompress_struct},Ptr{Cvoid},LibJpeg.JDIMENSION,LibJpeg.JDIMENSION,LibJpeg.boolean),
cinfo, unsafe_load(coef_arrays,ci), blk_y-1, 1, false))
for blk_x = 1:comp_info.width_in_blocks
bufptr = unsafe_load(buffer, blk_x)
for i=1:DCTSIZE, j=1:DCTSIZE
m[i + DCTSIZE*(blk_y-1), j + DCTSIZE*(blk_x-1)] =
bufptr[(i-1)*DCTSIZE + j]
end
end
end
push!(components, m)
end
LibJpeg.jpeg_destroy_decompress(cinfo)
close(io_file)
return components
end
jpeg_read_coefficients(filename::AbstractString) =
open(jpeg_read_coefficients, filename, "r")
It runs, in the sense that jpeg_read_coefficients(filename) returns an array of matrices of DCT coefficients (one per component). I’m not sure if it’s correct, since I don’t know what the right output is — it’s possible there is a typo, but at least the above code should be a good start.
Update: fixed to explicitly close the file as suggested by @pakk-minidose below
4 Likes
Incidentally, about 5 years ago, I wrote a JPEG (baseline) decoder as an exercise in learning Julia.
The original code doesn’t work with Julia v1 and the latest packages, so I’m trying to modernize it little by little, but I’ve been too busy lately. 
What I want to say is that obtaining DCT coefficients in pure Julia is not technically so difficult. 
2 Likes
Thank you everybody for your great replies!
However, when trying out @stevengj 's code, I run into the following error:
JPEG parameter struct mismatch: library thinks size is 600, caller expects 632
The test code was run using Julia 1.6.2 and the code was following:
using Pkg; Pkg.activate(temp=true) #create empty environment to isolate the issue
Pkg.add(url="https://github.com/stevengj/JpegTurbo.jl#main")
Pkg.add("TestImages")
#copy-paste from Discourse
import JpegTurbo: LibJpeg
import JpegTurbo.LibJpeg: DCTSIZE
function jpeg_read_coefficients(io::IO)
cinfo = LibJpeg.jpeg_decompress_struct()
jerr = Ref{LibJpeg.jpeg_error_mgr}()
cinfo.err = LibJpeg.jpeg_std_error(jerr)
LibJpeg.jpeg_create_decompress(cinfo)
LibJpeg.jpeg_stdio_src(cinfo, Libc.FILE(io))
LibJpeg.jpeg_read_header(cinfo, true)
coef_arrays = LibJpeg.jpeg_read_coefficients(cinfo)
access_virt_barray = unsafe_load(cinfo.mem).access_virt_barray
components = Matrix{LibJpeg.JCOEF}[]
for ci = 1:cinfo.num_components
comp_info = unsafe_load(cinfo.comp_info, ci)
c_height = comp_info.height_in_blocks * DCTSIZE
c_width = comp_info.width_in_blocks * DCTSIZE
m = Matrix{LibJpeg.JCOEF}(undef, c_height, c_width)
for blk_y = 1:comp_info.height_in_blocks
buffer = unsafe_load(ccall(access_virt_barray, LibJpeg.JBLOCKARRAY,
(Ref{LibJpeg.jpeg_decompress_struct},Ptr{Cvoid},LibJpeg.JDIMENSION,LibJpeg.JDIMENSION,LibJpeg.boolean),
cinfo, unsafe_load(coef_arrays,ci), blk_y-1, 1, false))
for blk_x = 1:comp_info.width_in_blocks
bufptr = unsafe_load(buffer, blk_x)
for i=1:DCTSIZE, j=1:DCTSIZE
m[i + DCTSIZE*(blk_y-1), j + DCTSIZE*(blk_x-1)] =
bufptr[(i-1)*DCTSIZE + j]
end
end
end
push!(components, m)
end
LibJpeg.jpeg_destroy_decompress(cinfo)
return components
end
jpeg_read_coefficients(filename::AbstractString) =
open(jpeg_read_coefficients, filename, "r")
#now the test itself
using TestImages
fname = testimage("earth_apollo17", download_only=true)
jpeg_read_coefficients(fname)
Info about my Julia installation:
julia> versioninfo()
Julia Version 1.6.2
Commit 1b93d53fc4 (2021-07-14 15:36 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Thank you for your help.
Probably some of the libjpeg types are different under Windows (your machine) vs Mac (my machine). Can you run
using Clang, JpegTurbo_jll
jconfig_h = joinpath(JpegTurbo_jll.artifact_dir, "include", "jconfig.h")
jmorecfg_h = joinpath(JpegTurbo_jll.artifact_dir, "include", "jmorecfg.h")
jpeglib_h = joinpath(JpegTurbo_jll.artifact_dir, "include", "jpeglib.h")
wc = init(; headers = [jconfig_h, jmorecfg_h, jpeglib_h],
output_file = joinpath(@__DIR__, "libjpeg_out.jl"),
common_file = joinpath(@__DIR__, "libjpeg_common.jl"),
header_wrapped = (root, current)->root == current,
header_library = x->"libjpeg"
)
run(wc)
on your machine and compare the libjpeg_common.jl file (with diff or similar) to the Mac output (at this gist).
Here is the diff. I below is a screenshot of the output as it is easier to read with its color coding
PS C:\tmp> git diff --no-index .\libjpeg_common_win.jl .\libjpeg_common_mac.jl
warning: LF will be replaced by CRLF in .\libjpeg_common_win.jl.
The file will have its original line endings in your working directory
warning: LF will be replaced by CRLF in .\libjpeg_common_mac.jl.
The file will have its original line endings in your working directory
diff --git "a/.\\libjpeg_common_win.jl" "b/.\\libjpeg_common_mac.jl"
index 6a645da..9452e09 100644
--- "a/.\\libjpeg_common_win.jl"
+++ "b/.\\libjpeg_common_mac.jl"
@@ -4,10 +4,17 @@
const JPEG_LIB_VERSION = 62
const LIBJPEG_TURBO_VERSION = ".1."
const LIBJPEG_TURBO_VERSION_NUMBER = 2001000
+const C_ARITH_CODING_SUPPORTED = 1
+const D_ARITH_CODING_SUPPORTED = 1
+const MEM_SRCDST_SUPPORTED = 1
+const WITH_SIMD = 1
const BITS_IN_JSAMPLE = 8
-const boolean = Cuchar
-const INT16 = Int16
-const INT32 = Cint
+const HAVE_LOCALE_H = 1
+const HAVE_STDDEF_H = 1
+const HAVE_STDLIB_H = 1
+const NEED_SYS_TYPES_H = 1
+const HAVE_UNSIGNED_CHAR = 1
+const HAVE_UNSIGNED_SHORT = 1
const MAX_COMPONENTS = 10
# Skipping MacroDefinition: GETJOCTET ( value ) ( value )
@@ -26,7 +33,10 @@ const JCOEF = Int16
const JOCTET = Cuchar
const UINT8 = Cuchar
const UINT16 = UInt32
+const INT16 = Int16
+const INT32 = Clong
const JDIMENSION = UInt32
+const boolean = Cint
const DCTSIZE = 8
const DCTSIZE2 = 64
const NUM_QUANT_TBLS = 4
Thank you.
It looks like the culprit is the boolean type, which is apparently unsigned char on Windows and int on Mac. I’ve pushed an update that should fix this.
2 Likes
Great! It now works on Windows. I’ve done a very coarse comparison with the MATLAB output and it seems to do the same.
Thank you for your help!
1 Like
Just adding a small bugfix.
It seems that the Julia Libc FILE does not release the file descriptor even though it is not reachable anymore, see this comment in the source code. I found out when my Julia instance ran out of file descriptors (Linux limits number of file descriptors per process to 1024) and the code above kept throwing SystemError: fdopen: Bad file descriptor in the Libc.FILE(io) call.
Here is a version which closes the file descriptor for anyone who would like to run the code
function jpeg_read_coefficients(io::IO)
cinfo = LibJpeg.jpeg_decompress_struct()
jerr = Ref{LibJpeg.jpeg_error_mgr}()
cinfo.err = LibJpeg.jpeg_std_error(jerr)
LibJpeg.jpeg_create_decompress(cinfo)
fle = Libc.FILE(io)
LibJpeg.jpeg_stdio_src(cinfo, fle)
LibJpeg.jpeg_read_header(cinfo, true)
coef_arrays = LibJpeg.jpeg_read_coefficients(cinfo)
access_virt_barray = unsafe_load(cinfo.mem).access_virt_barray
components = Matrix{LibJpeg.JCOEF}[]
for ci = 1:cinfo.num_components
comp_info = unsafe_load(cinfo.comp_info, ci)
c_height = comp_info.height_in_blocks * DCTSIZE
c_width = comp_info.width_in_blocks * DCTSIZE
m = Matrix{LibJpeg.JCOEF}(undef, c_height, c_width)
for blk_y = 1:comp_info.height_in_blocks
buffer = unsafe_load(ccall(access_virt_barray, LibJpeg.JBLOCKARRAY,
(Ref{LibJpeg.jpeg_decompress_struct},Ptr{Cvoid},LibJpeg.JDIMENSION,LibJpeg.JDIMENSION,LibJpeg.boolean),
cinfo, unsafe_load(coef_arrays,ci), blk_y-1, 1, false))
for blk_x = 1:comp_info.width_in_blocks
bufptr = unsafe_load(buffer, blk_x)
for i=1:DCTSIZE, j=1:DCTSIZE
m[i + DCTSIZE*(blk_y-1), j + DCTSIZE*(blk_x-1)] =
bufptr[(i-1)*DCTSIZE + j]
end
end
end
push!(components, m)
end
LibJpeg.jpeg_destroy_decompress(cinfo)
close(fle)
return components
end
jpeg_read_coefficients(filename::AbstractString) =
open(jpeg_read_coefficients, filename, "r")
5 posts were split to a new topic: Corrections to other posts: best practices?