Undefs.jl: Convenience and Experiment

Undefs.jl

I created a registered package called Undefs.jl that exports three names:

  1. undefs(A = Array, T = Float64, dims) = A{T}(undef, dims): a convenience method similar to ones, zeros, trues, and falses.
  2. undef!(array::Array, index=1): An experimental method to reset an element to an #undef state. This also works on Base.RefValue.
  3. @undef! A[1]: A macro to make the use of the undef! easier.

Demonstration of the convenience constructor, undefs

julia> using Undefs

julia> mutable struct Foo
           Foo(str) = finalizer(_->@async(println(str)), new())
       end

julia> undefs(5, 3)
5×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0

julia> undefs(Foo, 5, 3)
5×3 Matrix{Foo}:
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef

julia> A = undefs(Array, Foo, 5, 3)
5×3 Matrix{Foo}:
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef

julia> using OffsetArrays

julia> undefs(OffsetArray, UInt8, 5, 3)
5×3 OffsetArray(::Matrix{UInt8}, 1:5, 1:3) with eltype UInt8 with indices 1:5×1:3:
 0xf0  0x7f  0x00
 0x76  0x00  0x00
 0xf1  0x00  0x00
 0x3a  0x00  0x00
 0x3f  0x00  0x00

Demonstration of @undef!

julia> A[1] = Foo("bye world")
Foo()

julia> A
5×3 Matrix{Foo}:
    Foo()  #undef  #undef
 #undef    #undef  #undef
 #undef    #undef  #undef
 #undef    #undef  #undef
 #undef    #undef  #undef

julia> @undef! A[1]

julia> A
5×3 Matrix{Foo}:
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef

julia> GC.gc()
bye world

Discussion

Undefs.jl is a meant as experimental convenience package and is not meant for production. The convenience constructor, undefs is meant to advance the conversation from recurring calls for a simpler way to allocate arrays without initialization. Additionally, the function undef! and the macro @undef! are highly experimental and use internal details of Julia’s array implementation to unassign elements from arrays of mutable structures.

https://github.com/mkitti/Undefs.jl

While this package provides some convenient tools to play with, I encourage users to seek alternatives as described in the package README.

There’s a Base function for unsetting Array elements, if you don’t want to worry about the internal details yourself (which a prone to changing often). Undefing any other field by unsetting them is undefined behavior, and may lead to miscompilation or program crashes (pun intended)

This package is not meant to be taken too seriously. What’s the name of the Base function?

Do you mean the exported C function jl_arrayunset ?

Yes, but I think there is also a faster version of that C function in Julia

Base._unsetindex! ?

Yes

Is there a version intended for mutable structs or Base.RefValue?

No, the compiler relies on knowing it does not happen

I see. So array elements can be unset, but not elements of mutable structs like Base.RefValue?

correct

I encountered a similar problem. From this very interesting discussion, I understand that it is only safe to unset entries in Array (and Memory), not in other structures. I also found jl_arrayunset and Base._unsetindex! but this latter function is neither documented nor public which seems a bit odd as it is used by some Julia packages.

BTW, to solve my problem, I have written a small package, called UnsetIndex.jl, to unset array entries and which has some overlap with Undefs.

UnsetIndex exports two symbols:

  • a function unsetindex! which calls Base._indexunset! if it exists (Julia version ≥ 1.3) and otherwise calls the C function jl_arrayunset;
  • a constant unset that is a singleton of a special type for which Base.setindex! and other base methods are extended to achieve similar results as the @undef! macro and more but with a more natural syntax.

To unset an entry of array A at index i, do one of:

unsetindex!(A, i)
A[i] = unset
setindex!(A, unset, i)

The entry is left untouched if it has a bit type. In any case, bound checking is
performed unless @inbounds is active. The only restriction is that A must be of type
Array or Memory.

Dot notation is supported. For example:

julia> using UnsetIndex

julia> A = split("This is a wonderful world indeed!")
6-element Vector{SubString{String}}:
 "This"
 "is"
 "a"
 "wonderful"
 "world"
 "indeed!"

julia> A[end] = unset
#undef

julia> A
6-element Vector{SubString{String}}:
    "This"
    "is"
    "a"
    "wonderful"
    "world"
 #undef

julia> A[1:2:5] .= unset
2-element view(::Vector{SubString{String}}, 1:2:3) with eltype SubString{String}:
 #undef
 #undef
 #undef

julia> A
6-element Vector{SubString{String}}:
 #undef
    "is"
 #undef
    "wonderful"
 #undef
 #undef

julia> @. A = unset
6-element Vector{SubString{String}}:
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef

For the last example, A[:] .= unset works as well and I have checked that these two are as fast as explictly writing a loop over all the elements of A.

There is a feature request: