Makie - 3D volume rendering - using Axis3 - questions/issues

Hello, this is my first go at using Julia and GLMakie.
The goal is to display a 3D MIP CT image extracted from DICOM data.

Julia 1.6
DICOM v0.10.0
AbstractPlotting v0.17.3
GLMakie v0.2.6
Windows 10 x64
Visual Studio Code 1.55.2

Version 1 code:

module QntImageDisplay

using ModernGL
using GLMakie
using Colors

export static_image_display_v_1
function static_image_display_v_1(
            imageVolume::Array{Float32, 3}) 

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt3DMIPDisplay")

    set_theme!(backgroundcolor = :indigo)

    voxelScalarCutoffMin::Float32 = -200.0
    voxelScalarMax::Float32 = maximum(imageVolume)

    image3D = volume(
                imageVolume,  #  (rows, columns, slices) order
                algorithm=:mip,
                colormap=:grays,
                colorrange = (voxelScalarCutoffMin, voxelScalarMax))

    display(image3D)

end

end

3D_MP_Display_v_1_2021_04_19.jpg “Sorry, new users can only put one embedded media item in a post.”

This code works as expected. However, the spatial aspect is not correct - the patient looks like he has been subjected to the medieval rack. I next tried to use Axis3 to display the correct spatial aspect based upon my understanding of the Axis3 Makie documentation

Version 2 code:

module QntImageDisplay

using ModernGL
using GLMakie
using Colors

export static_image_display_v_2
function static_image_display_v_2(
            imageVolume::Array{Float32, 3},
            spatialExtent::Vector{Float32})

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt3DMIPDisplay")

    set_theme!(backgroundcolor = :indigo)

    figure = Figure(
                resolution = (1280, 720), 
                fontsize = 12)

    # In this example: spatialExtent Float32[380.416, 380.416, 401.0]

    axis3 = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                viewmode = :fitzoom)

    voxelScalarCutoffMin::Float32 = -200.0
    voxelScalarMax::Float32 = maximum(imageVolume)

    volume!(
        imageVolume,
        algorithm=:mip,
        colormap=:grays,
        colorrange = (voxelScalarCutoffMin, voxelScalarMax))

    display(figure)

end

end

Issues:

The 3D image spatial aspect is now correct. However, the mouse controls do not respond as expected.

  1. Right mouse button: no pan
  2. Middle mouse button: no zoom
  3. Left mouse button. Rotation restricted to half of one full rotation. Rotation appears to be convolved with zoom.

Is there a problem with my use of Figure and/or Axis3 in the above code or are these Axis3 issues?

Also, a general beginner’s question. To get GLMakie to display the output, I found that I had to run the top level file in the REPL as
julia> include(“QntMain.jl”)
rather than “Run code” of the .run extension. Is this a feature or am I missing something?

1 Like

The rotation is restricted so you can’t rotate across the top (then you would see things upside down which I always found a bit annoying with the normal 3D scenes). Horizontal rotation is not limited. Zooming would break the axis alignment so I currently don’t allow it, but might. Panning is the same way, the axis is always centered to its limit box. The rotation convolved with zoom is because of the viewmode, try :fit instead of :fitzoom Axis3 · Makie Plotting Ecosystem

After all, if you need more flexible camera position, maybe just use the old 3D axis via LScene or just scene? Although it of course doesn’t look as nice and doesn’t adjust to rotation

@Audrius-St Please tell us about the work you are doing with Julia. What interesting research are you doing, and how does Julia help you?

You need to give the volume the correct x-z dimensions:

volume(
    map(i-> 0..spatialExtent[i], 1:3)...,
    imageVolume,  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=:grays,
    colorrange = (voxelScalarCutoffMin, voxelScalarMax))

Fair point. Switching to viewmode = :fit, as below, this restriction makes sense.

Panning is not required, however, I suggest that zooming is an essential operation for 3D volume rendering.

Yes, this solves the issue. Thank you.

Axis3 is now sufficient and I assume this new method is the one to use going forward.

Thank you for your explanations.

A few follow-up cosmetic questions:

  1. How does one change the colour of the bounding box and the axes labels and numbers, from black to white?

  2. How does one change the orientation of the z axis label from horizontal to vertical?

The doc string of Axis3 lists all available attributes, just look there for color and label rotation and you should find the relevant names API Reference MakieLayout · Makie Plotting Ecosystem

Thank you for for the link.

with Axis3 as

axis3D = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                perspective = 0.5,
                viewmode = :fit,
                xgridvisible = false,
                ygridvisible = false,
                zgridvisible = false,
                xlabelcolor = :lightgray,
                ylabelcolor = :lightgray,
                zlabelcolor = :lightgray,
                xspinecolor = :lightgray,
                yspinecolor = :lightgray,                
                zspinecolor = :lightgray,
                xtickcolor = :lightgray,
                ytickcolor = :lightgray,
                ztickcolor = :lightgray,
                xticklabelcolor = :lightgray,
                yticklabelcolor = :lightgray,
                zticklabelcolor = :lightgray)

However, after several rotations the axes labels positions are off

I’ve seen that too, haven’t had time to investigate, yet

Understood. I’ve turned off all grid/label/spine/tick elements for now.

Thank you for this additional information.

That sounds like accumulated floating Point errors!

Btw, you should also be able to get rid of the black background by adding transparent values to the end of the colormap - if that wasn’t on purpose :wink:

1 Like

Indeed. How does one do this in Makie?
The Makie - Colors documentation mentions

The colorrange::NTuple{2,Number} attribute can be used to define the data values that correspond with the ends of the colormap.

but does not provide an example.
Would you please provide a code fragment example.

Specifically, I would like to add a transparent value to the lower end of the colormap and clamp values above the max of the colorrange to the upper color of the colormap.

using GLMakie
GLMakie.AbstractPlotting.inline!(false)
cmap = RGBAf0.(to_colormap(:viridis), 1.0)
cmap[1] = RGBAf0(1,1,1,0)

volume(
    rand(10, 10, 10),  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=cmap,
    figure=(backgroundcolor=:black,),
    transparency=true,
    colorrange = (0.5, 0.9)) |> display

You may also be interested in the output of contour(volume) :wink:

Thanks for your code fragment. I implement it, however, the output is as follows:

No effect. In my code below, the line

cmap[1] = RGBAf0(1, 1, 1, 0)

reports “Possible method called error.”

RGBAf0 is referred to in the AbstractPlotting documentation, although a “RGBAf0” keyword search does not return a specification, but not in the ColorTypes documentation which only specifies RGBA.

The current code

module QntImageDisplay

using ModernGL
using GLMakie
using Colors
using ColorSchemes
using ColorTypes

export static_image_display

function static_image_display(
            imageModality::String,
            imageVolume::Array{Float32, 3},
            spatialExtent::Vector{Float32})

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt 3D MIP Display")

    set_theme!(backgroundcolor = :black)

    figure = Figure(
                resolution = (1280, 720), 
                fontsize = 16)

    axis3D = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                perspective = 0.5,
                viewmode = :fit,
                titlevisible = false,
                xgridvisible = false,
                ygridvisible = false,
                zgridvisible = false,
                xlabelvisible = false,
                ylabelvisible = false,
                zlabelvisible = false,
                xspinesvisible = false,
                yspinesvisible = false,
                zspinesvisible = false,
                xticksvisible = false,
                yticksvisible = false,
                zticksvisible = false,
                xticklabelsvisible = false,
                yticklabelsvisible = false,
                zticklabelsvisible = false)

    if imageModality == "CT"
        voxelScalarCutoffMin::Float32 = max(-200.0, minimum(imageVolume)) # HU fat
        voxelScalarCutoffMax::Float32 = min(1900.0, maximum(imageVolume)) # HU bone

    end

    cmap = RGBAf0.(to_colormap(:viridis), 1.0)
    cmap[1] = RGBAf0(1, 1, 1, 0)

    image3D = volume!(
                imageVolume,
                algorithm = :mip,
                colormap = cmap,
                colorrange = (voxelScalarCutoffMin, voxelScalarCutoffMax),
                ssao = true,
                transparency = true) 

    display(figure)

end

end