Spot the error.. need help converting a c libturbojpeg example

General Usage
1

I’m trying to wrap this c example of libturbojpeg compression, but while tjCompress2() doesn’t error, its supposed to write into jpegBuf and set jpegSize to the length of jpegBuf but currently both are unchanged. It seems like tjCompress2() isn’t being passed the srcBuf properly, or jpegBuf isn’t initialized properly…

I’m following the docs here: libjpeg.turbojpeg source code

C example (from here):

#include "turbojpeg.h"
#include <iostream>
#include <string.h>
#include <errno.h>

using namespace std;

int main(void)
{
    unsigned char *srcBuf; //passed in as a param containing pixel data in RGB pixel interleaved format
    tjhandle handle = tjInitCompress();

    if(handle == NULL)
    {
        const char *err = (const char *) tjGetErrorStr();
        cerr << "TJ Error: " << err << " UNABLE TO INIT TJ Compressor Object\n";
        return -1;
    }
    int jpegQual =92;
    int width = 128;
    int height = 128;
    int nbands = 3;
    int flags = 0;
    unsigned char* jpegBuf = NULL;
    int pitch = width * nbands;
    int pixelFormat = TJPF_GRAY;
    int jpegSubsamp = TJSAMP_GRAY;
    if(nbands == 3)
    {
        pixelFormat = TJPF_RGB;
        jpegSubsamp = TJSAMP_411;
    }
    unsigned long jpegSize = 0;

    srcBuf = new unsigned char[width * height * nbands];
    for(int j = 0; j < height; j++)
    {
        for(int i = 0; i < width; i++)
        {
            srcBuf[(j * width + i) * nbands + 0] = (i) % 256;
            srcBuf[(j * width + i) * nbands + 1] = (j) % 256;
            srcBuf[(j * width + i) * nbands + 2] = (j + i) % 256;
        }
    }

    int tj_stat = tjCompress2( handle, srcBuf, width, pitch, height,
        pixelFormat, &(jpegBuf), &jpegSize, jpegSubsamp, jpegQual, flags);
    if(tj_stat != 0)
    {
        const char *err = (const char *) tjGetErrorStr();
        cerr << "TurboJPEG Error: " << err << " UNABLE TO COMPRESS JPEG IMAGE\n";
        tjDestroy(handle);
        handle = NULL;
        return -1;
    }

    FILE *file = fopen("out.jpg", "wb");
    if (!file) {
        cerr << "Could not open JPEG file: " << strerror(errno);
        return -1;
    }
    if (fwrite(jpegBuf, jpegSize, 1, file) < 1) {
        cerr << "Could not write JPEG file: " << strerror(errno);
        return -1;
    }
    fclose(file);

    //write out the compress date to the image file
    //cleanup
    int tjstat = tjDestroy(handle); //should deallocate data buffer
    handle = 0;
}

Julia attempt (with a simpler source image), using a Clang-generated wrapper (src here) :

using JpegTurbo_jll
libjpeg_wrap_dir = joinpath(@__DIR__, "..", "..", "gen", "libturbojpeg")
using CEnum
include(joinpath(libjpeg_wrap_dir, "ctypes.jl"))
export Ctm, Ctime_t, Cclock_t
include(joinpath(libjpeg_wrap_dir, "libturbojpeg_common.jl"))
include(joinpath(libjpeg_wrap_dir, "libturbojpeg_api.jl"))

using ColorTypes, ImageCore, FixedPointNumbers

img = rand(RGB{N0f8}, 10, 10)
rawimg = rawview(channelview(img))
srcBuf = collect(vec(rawimg))

nbands, width, height = size(rawimg)
pitch = Int(width * nbands)
pixelFormat = nbands == 3 ? TJPF_RGB : TJPF_GRAY
jpegSubsamp = nbands == 3 ? TJSAMP_411 : TJSAMP_GRAY
jpegQual = 92
flags = 0

handle = tjInitCompress()
handle == C_NULL && error("TurboJPEG Error: Unable to initialize compressor object: $(unsafe_string(tjGetErrorStr()))")

jpegBuf = Ptr{UInt8}(C_NULL)
jpegSize = Culong(0)

tj_stat = tjCompress2(handle, srcBuf, width, pitch, height, pixelFormat,
                    Ref(jpegBuf), Ref(jpegSize), jpegSubsamp, jpegQual, flags)

if tj_stat != 0
    err = tjGetErrorStr()
    tjDestroy(handle)
    handle = C_NULL
    error("TurboJPEG Error: $(unsafe_string(err))")
end
@show jpegSize

jpegBuf_j = unsafe_wrap(Array, jpegBuf, jpegSize; own = false)
open(joinpath(@__DIR__, "out.jpg"), "w") do io
   write(io, jpegBuf_j)
end

tjstat = tjDestroy(handle) #should deallocate data buffer
handle = nothing
tjFree(jpegBuf)

The @show jpegSize returns 0x0000000000000000, indicating that tjCompress2() hasn’t errored, but that no data has been written to jpegBuf

2

Found a workaround by pre-allocating the memory to the maximum possible size, as per the 1st option in the docs libjpeg.turbojpeg source code

    jpegSize = tjBufSize(width, height, jpegSubsamp)
    jpegBuf = tjAlloc(jpegSize)

It’s not ideal as it requires the allocation to be the max possible, which is actually larger than the source image size…

3

Hi Ian,

was this question part of the work on exposing libturbojpeg to julia?

4

Hi! Indeed. It’s been a while, but I was trying to make a wrapper to add JPEG io functionality to what now is ImageIO

Here’s some of the work in progress from back then https://github.com/ianshmean/ImageIODevelopment.jl

It’d be great to add JPEG to ImageIO if that’s what you’re working on!

5

Unfortunately, i do not have really a time. I still sometimes work on steganography, where an access to raw dct coefficients is a must. But I have too many things to work on. But it is good to know this exists, may be, one day i will have a bright student.

6

I’m encountering the same problem here, also with auto-generated Clang.jl wrappers using the JpegTurbo_jll package.

I’ve tracked down where libjpeg-turbo allocates the buffer to this code in jdatadst-tj.c:

if (*outbuffer == NULL || *outsize == 0) {
    if (alloc) {
      /* Allocate initial buffer */
      dest->newbuffer = *outbuffer = (unsigned char *)malloc(OUTPUT_BUF_SIZE);
      if (dest->newbuffer == NULL)
        ERREXIT1(cinfo, JERR_OUT_OF_MEMORY, 10);
      *outsize = OUTPUT_BUF_SIZE;
    } else
      ERREXIT(cinfo, JERR_BUFFER_SIZE);
  }

Is something going wrong with the pointer assignment here? Is it because we replace the address referenced by the pointer, and that messes up Julia’s C interop?

I don’t know C very well, so please correct me if I’m wrong.

Any help would be appreciated here!

7

I actually found a solution! Sort of…

I found two workarounds, one using Refs and another using Julia vectors:

include("TurboJpeg.jl") # Generated via Clang.jl
using Images

img = load("img_file.png")
# Convert to raw RGB pixel data
imgA = reshape(rawview(channelview(PermutedDimsArray(img, [2,1]))), *(size(img)..., 3)) |> collect

# Start interfacing with libjpeg-turbo
tjInstance = TurboJpeg.tjInitCompress() ; height, width = size(img)

# You have to specify Ref{Ptr{UInt8}} since the automatic type
# Ref{Ptr{Nothing}} fails to convert to Ptr{Ptr{UInt8}}
jpegBufPtr = Ref{Ptr{UInt8}}(C_NULL)
jpegSizePtr = Ref(Culong(0))

# Alternatively, you can also use Vectors:
# jpegBufPtr = [C_NULL]
# jpegSizePtr = [Culong(0)]

TurboJpeg.tjCompress2(tjInstance, imgA, width, width*3, height, TurboJpeg.TJPF_RGB, jpegBufPtr, jpegSizePtr, TurboJpeg.TJSAMP_420, 89, 0)

outbytes = unsafe_wrap(Vector{UInt8}, jpegBufPtr[], jpegSizePtr[])
# For the Vector version:
# outbytes = unsafe_wrap(Vector{UInt8}, jpegBufPtr[1], jpegSizePtr[1])

I’m not sure if this causes any memory leaks though since we aren’t calling tjFree() on the array once it’s passed back to Julia.

8

The use of Ref works slightly different (see e.g. this section in the docs), the comment above also uses this. Here’s an updated version of your example that works for me, I highlighted the changes:

using JpegTurbo_jll
libjpeg_wrap_dir = joinpath(@__DIR__, "..", "..", "gen", "libturbojpeg")
using CEnum
include(joinpath(libjpeg_wrap_dir, "ctypes.jl"))
export Ctm, Ctime_t, Cclock_t
include(joinpath(libjpeg_wrap_dir, "libturbojpeg_common.jl"))
include(joinpath(libjpeg_wrap_dir, "libturbojpeg_api.jl"))

using ColorTypes, ImageCore, FixedPointNumbers

img = rand(RGB{N0f8}, 10, 10)
rawimg = rawview(channelview(img))
srcBuf = collect(vec(rawimg))

nbands, width, height = size(rawimg)
pitch = Int(width * nbands)
pixelFormat = nbands == 3 ? TJPF_RGB : TJPF_GRAY
jpegSubsamp = nbands == 3 ? TJSAMP_411 : TJSAMP_GRAY
jpegQual = 92
flags = 0

handle = tjInitCompress()
handle == C_NULL && error("TurboJPEG Error: Unable to initialize compressor object: $(unsafe_string(tjGetErrorStr()))")

jpegBuf = Ptr{UInt8}(C_NULL)
jpegSize = Culong(0)

# Create the Ref wrappers, but make sure to save them
ptr_jpegBuf = Ref(jpegBuf)
ptr_jpegSize = Ref(jpegSize)

tj_stat = tjCompress2(handle, srcBuf, width, pitch, height, pixelFormat,
                    ptr_jpegBuf, ptr_jpegSize, jpegSubsamp, jpegQual, flags)

if tj_stat != 0
    err = tjGetErrorStr2()
    tjDestroy(handle)
    handle = C_NULL
    error("TurboJPEG Error: $(unsafe_string(err))")
end

# Get the actual value pointed to by the Ref, using the [] operator
@show ptr_jpegSize[]

# Same when retrieving the resulting image
jpegBuf_j = unsafe_wrap(Array, ptr_jpegBuf[], ptr_jpegSize[]; own = false)
open(joinpath(@__DIR__, "out.jpg"), "w") do io
   write(io, jpegBuf_j)
end

tjstat = tjDestroy(handle) #should deallocate data buffer
handle = nothing
tjFree(jpegBuf)

When running this I get:

melis@juggle 13:35:~/c/ImageIODevelopment.jl/src/jpeg$ j --project=../../ example_old.jl 
ptr_jpegSize[] = 0x000000000000036d

melis@juggle 13:35:~/c/ImageIODevelopment.jl/src/jpeg$ ls -l out.jpg 
-rw-r--r-- 1 melis users 877 Dec 27 13:35 out.jpg

Random 10x10 output image: image :slight_smile: