Q: No SubArray type required for passing partial multidimensional CuArrays?

Specific Domains GPU
1

Hello,

To pass part of a multidimensional array in Julia on the CPU on has to use the SubArray type in the function arguments.

This does not seem to be the case for Julia on the GPU.

Excerpted from the MWE below:

    @views image_display!(
            test_array[:, :, :, 3],
            test_cu_array[:, :, :, 3],
            a_title,
            kth_slice
        )

function image_display!(
            an_array::SubArray{Float64, 3},    # CPU: SubArray specified
            a_cu_array::CuArray{Float32, 3},  # GPU: No SubArray specified
            a_title::Vector{String},
            k_slc::Int64
        )

Before jumping to conclusions, as I could not find anything in the CUDA docs, I’d like to confirm if this is indeed the case;
that one can pass partial CuArrays as in the MApparentlyWE listed below:

# test_CUDA_SubArray_v1.jl

using ColorSchemes
using CUDA
using GLMakie
using Random

function image_display!(
            an_array::SubArray{Float64, 3}, 
            a_cu_array::CuArray{Float32, 3},
            a_title::Vector{String},
            k_slc::Int64
        )
    fig = Figure(size = (1920, 1080), figure_padding = 8, backgroundcolor = :gray80)

    grd_1_1 = fig[1, 1] = GridLayout()
    grd_1_2 = fig[1, 2] = GridLayout()

    axs_1_1 = 
        GLMakie.Axis(
            grd_1_1[1, 1],
            aspect = DataAspect(), title = string(a_title[1], "[:, :, ", string(k_slc), "]")
        )
    axs_1_2 = 
        GLMakie.Axis(
            grd_1_2[1, 1],
            aspect = DataAspect(), title = string(a_title[2], "[:, :, ", string(k_slc), "]")
        )

    image!(
        axs_1_1, an_array[:, :, k_slc], colormap = ColorSchemes.linear_grey_0_100_c0_n256, interpolate = false
    )
    image!(
        axs_1_2, 
        Array{Float64}(a_cu_array[:, :, k_slc]), colormap = ColorSchemes.linear_grey_0_100_c0_n256, interpolate = false
    )
    wait(Makie.display(fig, scalefactor = 3.0))
end

function main()
    n_rows = 256
    n_cols = 256
    n_slcs = 192

    # Test array
    test_array = Array{Float64, 4}(undef, (n_rows, n_cols, n_slcs, 3))
    rand!(test_array)

    # CUDA test array
    test_cu_array = CuArray{Float32}(test_array)
    @show typeof(test_cu_array)
    println("")

    # Debug
    kth_slice = 96
    a_title = ["test CPU SubArray", "test GPU array"]

    @views image_display!(
            test_array[:, :, :, 1],
            test_cu_array[:, :, :, 1],
            a_title,
            kth_slice
        )

    @views image_display!(
            test_array[:, :, :, 2],
            test_cu_array[:, :, :, 2],
            a_title,
            kth_slice
        )

    @views image_display!(
            test_array[:, :, :, 3],
            test_cu_array[:, :, :, 3],
            a_title,
            kth_slice
        )
end

begin
    main()
end
2

Hi,

As far as I understand, a CuArray is basically a pointer to some memory, together with an offset and size information (+ extra information for GC, etc.). If you take a contiguous view, you can represent this using the same memory pointer, with just a different offset and size. In particular, you don’t need a different type than CuArray. E.g.

julia> x = CUDA.rand(10, 10);

julia> x.data.rc.obj.mem.ptr  # (I'm not 100% this is actually the pointer I'm describing above, but it sounds very plausible and fits the narrative :) )
CuPtr{Nothing}(0x0000000204c00400)

julia> x.dims
(10, 10)

julia> x.offset
0

julia> v = view(x, 2:5, 2); typeof(v)
CuArray{Float32, 1, CUDA.DeviceMemory}

julia> v.data.rc.obj.mem.ptr  # same as for x
CuPtr{Nothing}(0x0000000204c00400)

julia> v.dims
(4,)

julia> v.offset  # 10 (== length(x[:, 1])) + 1 (== length(x[1, 2]))
11

If you take a non-contiguous view, you do end up with a SubArray:

julia> w = view(x, 2, 2:5); typeof(w)
SubArray{Float32, 1, CuArray{Float32, 2, CUDA.DeviceMemory}, Tuple{Int64, UnitRange{Int64}}, true}

For this reason (and also just for generality) I would advise to simply not explicitly specify the type in the signature of image_display!.

3

Thank you for your reply and explanation.

If I understand correctly, in the case of CuArrays the recommendation is to not explicitly specify the type in the signature of a function.

This seems to be an exception to the multiple dispatch paradigm of Julia.

4

You’re most likely misunderstanding multiple dispatch. This only means that whenever you have a call f(a, b) Julia will use the most specific method of f with respects of the types of a and b. If we only have a single method image_display!, then this is the only one which can get invoked, regardless of how we choose its signature.

The only advantage (here) to having a ::CuArray{Float32, 3} is that you throw a MethodError if someone would supply a different type. This might be clearer than whatever error you would get in the function body if you’re working with the unexpected type. But possibly there is no error and everything just works out fine. E.g. if typeof(an_array) === Array{Float64, 3} no issues will arise if you don’t specify the type (or keep it more general, like ::AbstractArray{T, 3} where T), so why artificially restrict ourselves to SubArray{Float64, 3}?

Also, to be clear, once the appropriate method of the function you’re calling is selected, the arguments have concrete types and the code is (or has been) just-in-time compiled for these types. So not specifying the types of arguments does not come with a performance penalty.

See also the section Avoid writing overly-specific types in the Style guide.