Makie - 3D volume rendering - using Axis3 - questions/issues

Audrius-St · April 19, 2021, 8:31pm

Hello, this is my first go at using Julia and GLMakie.
The goal is to display a 3D MIP CT image extracted from DICOM data.

Julia 1.6
DICOM v0.10.0
AbstractPlotting v0.17.3
GLMakie v0.2.6
Windows 10 x64
Visual Studio Code 1.55.2

Version 1 code:

module QntImageDisplay

using ModernGL
using GLMakie
using Colors

export static_image_display_v_1
function static_image_display_v_1(
            imageVolume::Array{Float32, 3}) 

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt3DMIPDisplay")

    set_theme!(backgroundcolor = :indigo)

    voxelScalarCutoffMin::Float32 = -200.0
    voxelScalarMax::Float32 = maximum(imageVolume)

    image3D = volume(
                imageVolume,  #  (rows, columns, slices) order
                algorithm=:mip,
                colormap=:grays,
                colorrange = (voxelScalarCutoffMin, voxelScalarMax))

    display(image3D)

end

end

3D_MP_Display_v_1_2021_04_19.jpg “Sorry, new users can only put one embedded media item in a post.”

This code works as expected. However, the spatial aspect is not correct - the patient looks like he has been subjected to the medieval rack. I next tried to use Axis3 to display the correct spatial aspect based upon my understanding of the Axis3 Makie documentation

Version 2 code:

module QntImageDisplay

using ModernGL
using GLMakie
using Colors

export static_image_display_v_2
function static_image_display_v_2(
            imageVolume::Array{Float32, 3},
            spatialExtent::Vector{Float32})

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt3DMIPDisplay")

    set_theme!(backgroundcolor = :indigo)

    figure = Figure(
                resolution = (1280, 720), 
                fontsize = 12)

    # In this example: spatialExtent Float32[380.416, 380.416, 401.0]

    axis3 = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                viewmode = :fitzoom)

    voxelScalarCutoffMin::Float32 = -200.0
    voxelScalarMax::Float32 = maximum(imageVolume)

    volume!(
        imageVolume,
        algorithm=:mip,
        colormap=:grays,
        colorrange = (voxelScalarCutoffMin, voxelScalarMax))

    display(figure)

end

end

Issues:

The 3D image spatial aspect is now correct. However, the mouse controls do not respond as expected.

Right mouse button: no pan
Middle mouse button: no zoom
Left mouse button. Rotation restricted to half of one full rotation. Rotation appears to be convolved with zoom.

Is there a problem with my use of Figure and/or Axis3 in the above code or are these Axis3 issues?

Also, a general beginner’s question. To get GLMakie to display the output, I found that I had to run the top level file in the REPL as
julia> include(“QntMain.jl”)
rather than “Run code” of the .run extension. Is this a feature or am I missing something?

jules · April 20, 2021, 5:49am

The rotation is restricted so you can’t rotate across the top (then you would see things upside down which I always found a bit annoying with the normal 3D scenes). Horizontal rotation is not limited. Zooming would break the axis alignment so I currently don’t allow it, but might. Panning is the same way, the axis is always centered to its limit box. The rotation convolved with zoom is because of the viewmode, try :fit instead of :fitzoom http://makie.juliaplots.org/stable/makielayout/axis3.html#viewmode

After all, if you need more flexible camera position, maybe just use the old 3D axis via LScene or just scene? Although it of course doesn’t look as nice and doesn’t adjust to rotation

johnh · April 20, 2021, 7:14am

@Audrius-St Please tell us about the work you are doing with Julia. What interesting research are you doing, and how does Julia help you?

sdanisch · April 20, 2021, 10:30am

You need to give the volume the correct x-z dimensions:

volume(
    map(i-> 0..spatialExtent[i], 1:3)...,
    imageVolume,  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=:grays,
    colorrange = (voxelScalarCutoffMin, voxelScalarMax))

Audrius-St · April 20, 2021, 7:55pm

Fair point. Switching to viewmode = :fit, as below, this restriction makes sense.

Panning is not required, however, I suggest that zooming is an essential operation for 3D volume rendering.

Yes, this solves the issue. Thank you.

Axis3 is now sufficient and I assume this new method is the one to use going forward.

Thank you for your explanations.

A few follow-up cosmetic questions:

How does one change the colour of the bounding box and the axes labels and numbers, from black to white?
How does one change the orientation of the z axis label from horizontal to vertical?

jules · April 20, 2021, 8:08pm

The doc string of Axis3 lists all available attributes, just look there for color and label rotation and you should find the relevant names http://makie.juliaplots.org/stable/makielayout/reference.html#AbstractPlotting.MakieLayout.Axis3

Audrius-St · April 21, 2021, 12:55am

Thank you for for the link.

with Axis3 as

axis3D = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                perspective = 0.5,
                viewmode = :fit,
                xgridvisible = false,
                ygridvisible = false,
                zgridvisible = false,
                xlabelcolor = :lightgray,
                ylabelcolor = :lightgray,
                zlabelcolor = :lightgray,
                xspinecolor = :lightgray,
                yspinecolor = :lightgray,                
                zspinecolor = :lightgray,
                xtickcolor = :lightgray,
                ytickcolor = :lightgray,
                ztickcolor = :lightgray,
                xticklabelcolor = :lightgray,
                yticklabelcolor = :lightgray,
                zticklabelcolor = :lightgray)

However, after several rotations the axes labels positions are off

jules · April 21, 2021, 3:53am

I’ve seen that too, haven’t had time to investigate, yet

Audrius-St · April 21, 2021, 6:05am

Understood. I’ve turned off all grid/label/spine/tick elements for now.

Audrius-St · April 21, 2021, 6:52am

sdanisch:

You need to give the volume the correct x-z dimensions:

volume(
    map(i-> 0..spatialExtent[i], 1:3)...,
    imageVolume,  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=:grays,
    colorrange = (voxelScalarCutoffMin, voxelScalarMax))

Thank you for this additional information.

sdanisch · April 21, 2021, 9:54am

That sounds like accumulated floating Point errors!

sdanisch · April 21, 2021, 11:16am

Btw, you should also be able to get rid of the black background by adding transparent values to the end of the colormap - if that wasn’t on purpose

Audrius-St · April 21, 2021, 5:52pm

Indeed. How does one do this in Makie?
The Makie - Colors documentation mentions

The colorrange::NTuple{2,Number} attribute can be used to define the data values that correspond with the ends of the colormap.

but does not provide an example.
Would you please provide a code fragment example.

Specifically, I would like to add a transparent value to the lower end of the colormap and clamp values above the max of the colorrange to the upper color of the colormap.

sdanisch · April 22, 2021, 10:27am

using GLMakie
GLMakie.AbstractPlotting.inline!(false)
cmap = RGBAf0.(to_colormap(:viridis), 1.0)
cmap[1] = RGBAf0(1,1,1,0)

volume(
    rand(10, 10, 10),  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=cmap,
    figure=(backgroundcolor=:black,),
    transparency=true,
    colorrange = (0.5, 0.9)) |> display

You may also be interested in the output of contour(volume)

Audrius-St · April 22, 2021, 7:56pm

sdanisch:

using GLMakie
GLMakie.AbstractPlotting.inline!(false)
cmap = RGBAf0.(to_colormap(:viridis), 1.0)
cmap[1] = RGBAf0(1,1,1,0)

volume(
    rand(10, 10, 10),  #  (rows, columns, slices) order
    algorithm=:mip,
    colormap=cmap,
    figure=(backgroundcolor=:black,),
    transparency=true,
    colorrange = (0.5, 0.9)) |> display

Thanks for your code fragment. I implement it, however, the output is as follows:

No effect. In my code below, the line

cmap[1] = RGBAf0(1, 1, 1, 0)

reports “Possible method called error.”

RGBAf0 is referred to in the AbstractPlotting documentation, although a “RGBAf0” keyword search does not return a specification, but not in the ColorTypes documentation which only specifies RGBA.

The current code

module QntImageDisplay

using ModernGL
using GLMakie
using Colors
using ColorSchemes
using ColorTypes

export static_image_display

function static_image_display(
            imageModality::String,
            imageVolume::Array{Float32, 3},
            spatialExtent::Vector{Float32})

    set_window_config!(
        float=true,
        framerate = 60.0,
        title = "Qnt 3D MIP Display")

    set_theme!(backgroundcolor = :black)

    figure = Figure(
                resolution = (1280, 720), 
                fontsize = 16)

    axis3D = Axis3(
                figure[1, 1],
                aspect = (
                    spatialExtent[1], 
                    spatialExtent[2], 
                    spatialExtent[3]),
                perspective = 0.5,
                viewmode = :fit,
                titlevisible = false,
                xgridvisible = false,
                ygridvisible = false,
                zgridvisible = false,
                xlabelvisible = false,
                ylabelvisible = false,
                zlabelvisible = false,
                xspinesvisible = false,
                yspinesvisible = false,
                zspinesvisible = false,
                xticksvisible = false,
                yticksvisible = false,
                zticksvisible = false,
                xticklabelsvisible = false,
                yticklabelsvisible = false,
                zticklabelsvisible = false)

    if imageModality == "CT"
        voxelScalarCutoffMin::Float32 = max(-200.0, minimum(imageVolume)) # HU fat
        voxelScalarCutoffMax::Float32 = min(1900.0, maximum(imageVolume)) # HU bone

    end

    cmap = RGBAf0.(to_colormap(:viridis), 1.0)
    cmap[1] = RGBAf0(1, 1, 1, 0)

    image3D = volume!(
                imageVolume,
                algorithm = :mip,
                colormap = cmap,
                colorrange = (voxelScalarCutoffMin, voxelScalarCutoffMax),
                ssao = true,
                transparency = true) 

    display(figure)

end

end

Topic		Replies	Views
Positioning a volume with Makie Visualization question , makie	2	452	January 19, 2021
Makie inconsistency between GLMakie output and saved mp4 output Visualization question , plotting , makie	0	274	August 1, 2022
Animate Makie.volume! doesn't work New to Julia plotting	4	75	November 27, 2024
[Makie.jl] 3D voxelized visualization of a cubical matrix Visualization makie	25	2552	July 25, 2024
Issue of volume render with large 3D data Visualization makie , 3d , glmakie	3	339	September 2, 2023

Makie - 3D volume rendering - using Axis3 - questions/issues

Related topics