Makie - 3D volume rendering - using Axis3 - questions/issues

Specific Domains Visualization
,
1

Hello, this is my first go at using Julia and GLMakie.
The goal is to display a 3D MIP CT image extracted from DICOM data.

Julia 1.6
DICOM v0.10.0
AbstractPlotting v0.17.3
GLMakie v0.2.6
Windows 10 x64
Visual Studio Code 1.55.2

Version 1 code:

module QntImageDisplay

using ModernGL
using GLMakie
using Colors

export static_image_display_v_1
function static_image_display_v_1(
            imageVolume::Array{Float32, 3}) 

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt3DMIPDisplay")

    set_theme!(backgroundcolor = :indigo)

    voxelScalarCutoffMin::Float32 = -200.0
    voxelScalarMax::Float32 = maximum(imageVolume)

    image3D = volume(
                imageVolume,  #  (rows, columns, slices) order
                algorithm=:mip,
                colormap=:grays,
                colorrange = (voxelScalarCutoffMin, voxelScalarMax))

    display(image3D)

end

end

3D_MP_Display_v_1_2021_04_19.jpg “Sorry, new users can only put one embedded media item in a post.”

This code works as expected. However, the spatial aspect is not correct - the patient looks like he has been subjected to the medieval rack. I next tried to use Axis3 to display the correct spatial aspect based upon my understanding of the Axis3 Makie documentation

Version 2 code:

module QntImageDisplay

using ModernGL
using GLMakie
using Colors

export static_image_display_v_2
function static_image_display_v_2(
            imageVolume::Array{Float32, 3},
            spatialExtent::Vector{Float32})

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt3DMIPDisplay")

    set_theme!(backgroundcolor = :indigo)

    figure = Figure(
                resolution = (1280, 720), 
                fontsize = 12)

    # In this example: spatialExtent Float32[380.416, 380.416, 401.0]

    axis3 = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                viewmode = :fitzoom)

    voxelScalarCutoffMin::Float32 = -200.0
    voxelScalarMax::Float32 = maximum(imageVolume)

    volume!(
        imageVolume,
        algorithm=:mip,
        colormap=:grays,
        colorrange = (voxelScalarCutoffMin, voxelScalarMax))

    display(figure)

end

end

3D_MIP_Display_v_2_2021_04_19
3D_MIP_Display_v_2_2021_04_191282Ă—752 60.8 KB

Issues:

The 3D image spatial aspect is now correct. However, the mouse controls do not respond as expected.

  1. Right mouse button: no pan
  2. Middle mouse button: no zoom
  3. Left mouse button. Rotation restricted to half of one full rotation. Rotation appears to be convolved with zoom.

Is there a problem with my use of Figure and/or Axis3 in the above code or are these Axis3 issues?

Also, a general beginner’s question. To get GLMakie to display the output, I found that I had to run the top level file in the REPL as
julia> include(“QntMain.jl”)
rather than “Run code” of the .run extension. Is this a feature or am I missing something?

1 Like
2

The rotation is restricted so you can’t rotate across the top (then you would see things upside down which I always found a bit annoying with the normal 3D scenes). Horizontal rotation is not limited. Zooming would break the axis alignment so I currently don’t allow it, but might. Panning is the same way, the axis is always centered to its limit box. The rotation convolved with zoom is because of the viewmode, try :fit instead of :fitzoom http://makie.juliaplots.org/stable/makielayout/axis3.html#viewmode

After all, if you need more flexible camera position, maybe just use the old 3D axis via LScene or just scene? Although it of course doesn’t look as nice and doesn’t adjust to rotation

3

@Audrius-St Please tell us about the work you are doing with Julia. What interesting research are you doing, and how does Julia help you?

4

You need to give the volume the correct x-z dimensions:

volume(
    map(i-> 0..spatialExtent[i], 1:3)...,
    imageVolume,  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=:grays,
    colorrange = (voxelScalarCutoffMin, voxelScalarMax))
5

Fair point. Switching to viewmode = :fit, as below, this restriction makes sense.

Panning is not required, however, I suggest that zooming is an essential operation for 3D volume rendering.

Yes, this solves the issue. Thank you.

3D_MIP_Display_v_3_2021_04_20
3D_MIP_Display_v_3_2021_04_201922Ă—1089 129 KB

Axis3 is now sufficient and I assume this new method is the one to use going forward.

Thank you for your explanations.

A few follow-up cosmetic questions:

  1. How does one change the colour of the bounding box and the axes labels and numbers, from black to white?

  2. How does one change the orientation of the z axis label from horizontal to vertical?

6

The doc string of Axis3 lists all available attributes, just look there for color and label rotation and you should find the relevant names http://makie.juliaplots.org/stable/makielayout/reference.html#AbstractPlotting.MakieLayout.Axis3

7

Thank you for for the link.

3D_MIP_Display_v_4_2021_04_20
3D_MIP_Display_v_4_2021_04_201922Ă—1089 124 KB

with Axis3 as

axis3D = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                perspective = 0.5,
                viewmode = :fit,
                xgridvisible = false,
                ygridvisible = false,
                zgridvisible = false,
                xlabelcolor = :lightgray,
                ylabelcolor = :lightgray,
                zlabelcolor = :lightgray,
                xspinecolor = :lightgray,
                yspinecolor = :lightgray,                
                zspinecolor = :lightgray,
                xtickcolor = :lightgray,
                ytickcolor = :lightgray,
                ztickcolor = :lightgray,
                xticklabelcolor = :lightgray,
                yticklabelcolor = :lightgray,
                zticklabelcolor = :lightgray)

However, after several rotations the axes labels positions are off

3D_MIP_Display_v_4_post_rotations_2021_04_20
3D_MIP_Display_v_4_post_rotations_2021_04_201922Ă—1089 113 KB

8

I’ve seen that too, haven’t had time to investigate, yet

9

Understood. I’ve turned off all grid/label/spine/tick elements for now.

10

Thank you for this additional information.

11

That sounds like accumulated floating Point errors!

12

Btw, you should also be able to get rid of the black background by adding transparent values to the end of the colormap - if that wasn’t on purpose :wink:

1 Like
13

Indeed. How does one do this in Makie?
The Makie - Colors documentation mentions

The colorrange::NTuple{2,Number} attribute can be used to define the data values that correspond with the ends of the colormap.

but does not provide an example.
Would you please provide a code fragment example.

Specifically, I would like to add a transparent value to the lower end of the colormap and clamp values above the max of the colorrange to the upper color of the colormap.

14 
using GLMakie
GLMakie.AbstractPlotting.inline!(false)
cmap = RGBAf0.(to_colormap(:viridis), 1.0)
cmap[1] = RGBAf0(1,1,1,0)

volume(
    rand(10, 10, 10),  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=cmap,
    figure=(backgroundcolor=:black,),
    transparency=true,
    colorrange = (0.5, 0.9)) |> display

You may also be interested in the output of contour(volume) :wink:

1 Like
15

Thanks for your code fragment. I implement it, however, the output is as follows:

3D_MIP_Display_v_5_2021_04_22
3D_MIP_Display_v_5_2021_04_221922Ă—1089 99.9 KB

No effect. In my code below, the line

cmap[1] = RGBAf0(1, 1, 1, 0)

reports “Possible method called error.”

RGBAf0 is referred to in the AbstractPlotting documentation, although a “RGBAf0” keyword search does not return a specification, but not in the ColorTypes documentation which only specifies RGBA.

The current code

module QntImageDisplay

using ModernGL
using GLMakie
using Colors
using ColorSchemes
using ColorTypes

export static_image_display

function static_image_display(
            imageModality::String,
            imageVolume::Array{Float32, 3},
            spatialExtent::Vector{Float32})

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt 3D MIP Display")

    set_theme!(backgroundcolor = :black)

    figure = Figure(
                resolution = (1280, 720), 
                fontsize = 16)

    axis3D = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                perspective = 0.5,
                viewmode = :fit,
                titlevisible = false,
                xgridvisible = false,
                ygridvisible = false,
                zgridvisible = false,
                xlabelvisible = false,
                ylabelvisible = false,
                zlabelvisible = false,
                xspinesvisible = false,
                yspinesvisible = false,
                zspinesvisible = false,
                xticksvisible = false,
                yticksvisible = false,
                zticksvisible = false,
                xticklabelsvisible = false,
                yticklabelsvisible = false,
                zticklabelsvisible = false)

    if imageModality == "CT"
        voxelScalarCutoffMin::Float32 = max(-200.0, minimum(imageVolume)) # HU fat
        voxelScalarCutoffMax::Float32 = min(1900.0, maximum(imageVolume)) # HU bone

    end

    cmap = RGBAf0.(to_colormap(:viridis), 1.0)
    cmap[1] = RGBAf0(1, 1, 1, 0)

    image3D = volume!(
                imageVolume,
                algorithm = :mip,
                colormap = cmap,
                colorrange = (voxelScalarCutoffMin, voxelScalarCutoffMax),
                ssao = true,
                transparency = true) 

    display(figure)

end

end