Debugging with ImageView.jl

New to Julia
,
1

I really like the simplicity of ImageView.jl. It appears to be everything I need:

  • showing an Matrix{Float64} as an image that scales appropriately,
  • annotating with simple shapes or text

I am currently working on a detection algorithm, and I want to show the current image with my detections boxed. I do not want to have to open and close a gui every time a new image comes, but rather update the image and the annotaitons at each frame. This is a very simple task in C++ and Matlab. I would have thought this would be a basic behavior to ImageView.jl. There must be something simple I am missing, but I do not seem to see an easy solution.

Thanks for any assistance!

using Statistics, Images, ImageView
function ex()
    numCols, numRows, numFrames = 100, 100, 100
    frameRate = 1 # numFrames / sec
    p0 = [25, 25] # initial position of target
    v = [1, 1] #  velocity pixel / sec
    ## Create data for reproducibility
    img = randn(numCols, numRows, numFrames) # creat white noise background
    kernel = 10 .* [0 1 0;
                    1 2 1;
                    0 1 0] # simple bright target
    p_truth = zeros(2, numFrames)
    for fn = 1:numFrames
        p = p0 + (fn / frameRate-1) * v # constant velocity
        y, x = Int(round(p[1])), Int(round(p[2]))  # convert position to nearest pixel for simplicity
        if 1 < y < numCols && 1 < x < numRows # our kernel is 3x3 so only get within one of the edge
            img[y-1:y+1, x-1:x+1, fn] = kernel
        end
        p_truth[:,fn] = p
    end

    ## Below is the code that I wish worked
    gd, c = nothing, nothing
    for fn in 1:numFrames
        if fn ==1
            gd = imshow(img[:,:,fn])
            c = gd["gui"]["canvas"]
        else
            # this part seems to be creating issues
            # if I use imshow! the matrix does not get converted in to the proper type, maybe ?
                imshow!(c, img[:,:,fn])
                # error message
                # Warning: Error in @guarded callback
                # └ @ GtkReactive ~/.julia/packages/Gtk/C22jV/src/base.jl:90
                # ::ERROR: ReactiveArgumentError: element type Normed{UInt8,8} is an 
                # 8-bit type representing 256 values from 0.0 to 1.0,
                #   but the values (-1.5032452089414814,) do not lie within this range.
                #   See the READMEs for FixedPointNumbers and ColorTypes for more information..
            # if I use imshow (sans !) this image changes, but zooming does not work
            # and annotations dont show up
                # imshow(c, img[:,:,fn])
        end
        # Get detections from algo
        list_of_detections = [p_truth[:,fn]] # for simplicity only grab true detections
        aIx = []
        for p in list_of_detections
            y, x = Int(round(p[1])), Int(round(p[2]))
            # Creating annotation do not show after first frame
            a = annotate!(gd,
                        AnnotationBox(x - 5, y - 5,
                                        x + 5, y + 5,
                                        linewidth=2, color=RGB(1,0,0))
                        )
            push!(aIx, a)
        end
        readline()
        [delete!(gd,a) for a in aIx] # remove previous detections after they have been seen
    end
end
ex()
2

@tim.holy do you have any suggestions?

3

?imshow; the example at the end gives a demo (see that final push!).

1 Like
4

I have tried this as well. The annotations are not connnected to the gui that is created by calling imshow in this way. Thanks for your help

    ## Below is the code that I wish worked
    # Change the type of the matrix to be compatible
    img .-= minimum(img)
    img = n0f8.(img./maximum(img)) # make the type of img be compatible with ImageView.Signal
    # Follow the example from the help string for imshow
    gd = imshow_gui((numCols, numRows))
    c = gd["canvas"]
    Gtk.showall(gd["window"])
    sig = ImageView.Signal(img[:,:,1])
    anns = annotations() # I do not see a Annotations in any of the fields of either roi_dict or gd
    roi_dict = imshow(c, sig)
    for fn in 2:numFrames
        push!(sig, img[:,:,fn]) # this works, and zoom is still supported
        # Get detections from algo
        list_of_detections = [p_truth[:,fn]] # for simplicity only grab true detections
        for p in list_of_detections
            y, x = Int(round(p[1])), Int(round(p[2]))
            # while this does not error out, no annotations are shown
            annotate!(anns, c, roi_dict,
                        AnnotationBox(x - 5, y - 5,
                                      x + 5, y + 5,
                                      linewidth=2, color=RGB(1,0,0))
                        ) 
            push!(aIx, a) 
        end
        sleep(0.5)
        [delete!(gd,a) for a in aIx] # remove previous detections after they have been seen
    end
5

And are you happy with the outcome?

For the purposes of your original question, the ImageView.closeall seems undesirable?

6

I am sorry, the close all should not appear anymore. I replaced it with a sleep(0.5).
The problem is that the annotations are not showing even though there is no error.

7

Here is the full solution. You have to include the annotations in the creation of the roi_dict. It took some digging but you also need give a zr::Signal{ZoomRegion{T}} to use the desired version of imshow. To get an out of the box solution, I needed to also inclued GtkReactive.

using Statistics, Images, ImageView, Gtk.ShortNames, GtkReactive
function ex()
    numCols, numRows, numFrames = 100, 100, 100
    frameRate = 1 # numFrames / sec
    p0 = [25, 25] # initial position of target
    v = [1, 1] #  velocity pixel / sec
    img = randn(numCols, numRows, numFrames) # creat white noise background
    kernel = 10 .* [0 1 0;
                    1 2 1;
                    0 1 0] # simple bright target
    p_truth = zeros(2, numFrames)
    for fn = 1:numFrames
        p = p0 + (fn / frameRate-1) * v # constant velocity
        y, x = Int(round(p[1])), Int(round(p[2]))  # convert position to nearest pixel for simplicity
        if 1 < y < numCols && 1 < x < numRows # our kernel is 3x3 so only get within one of the edge
            img[y-1:y+1, x-1:x+1, fn] = kernel
        end
        p_truth[:,fn] = p
    end
    img .-= minimum(img)
    img = n0f8.(img./maximum(img))
    gd = imshow_gui((numCols, numRows))
    c = gd["canvas"]
    Gtk.showall(gd["window"])
    sig = ImageView.Signal(img[:,:,1])
    anns = annotations()
    roi_dict = imshow(c, sig, Signal(ZoomRegion(value(sig))), anns)
    for fn in 2:numFrames
        push!(sig, img[:,:,fn]) # this works, and zoom is still supported
        # Get detections from algo
        list_of_detections = [p_truth[:,fn]] # for simplicity only grab true detections
        aIx = []
        for p in list_of_detections
            y, x = Int(round(p[1])), Int(round(p[2]))
            # Creating annotation do not show after first frame
            a = annotate!(anns, c, roi_dict,
                        AnnotationBox(x - 5, y - 5,
                                      x + 5, y + 5,
                                      linewidth=2, color=RGB(1,0,0))
                        )
            push!(aIx, a)
        end
        sleep(0.5)
        [delete!(anns,a) for a in aIx] # remove previous detections after they have been seen
    end
end
ex()
1 Like
8

I’m short on time so won’t dig into this since you’ve got a solution you’re happy with, but please do submit any issues (or better, PRs :wink: ) about what was missing in the documentation or missing bits of functionality that would have made this easier.