Undefs.jl: Convenience and Experiment

Undefs.jl

I created a registered package called Undefs.jl that exports three names:

  1. undefs(A = Array, T = Float64, dims) = A{T}(undef, dims): a convenience method similar to ones, zeros, trues, and falses.
  2. undef!(array::Array, index=1): An experimental method to reset an element to an #undef state. This also works on Base.RefValue.
  3. @undef! A[1]: A macro to make the use of the undef! easier.

Demonstration of the convenience constructor, undefs

julia> using Undefs

julia> mutable struct Foo
           Foo(str) = finalizer(_->@async(println(str)), new())
       end

julia> undefs(5, 3)
5×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0

julia> undefs(Foo, 5, 3)
5×3 Matrix{Foo}:
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef

julia> A = undefs(Array, Foo, 5, 3)
5×3 Matrix{Foo}:
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef

julia> using OffsetArrays

julia> undefs(OffsetArray, UInt8, 5, 3)
5×3 OffsetArray(::Matrix{UInt8}, 1:5, 1:3) with eltype UInt8 with indices 1:5×1:3:
 0xf0  0x7f  0x00
 0x76  0x00  0x00
 0xf1  0x00  0x00
 0x3a  0x00  0x00
 0x3f  0x00  0x00

Demonstration of @undef!

julia> A[1] = Foo("bye world")
Foo()

julia> A
5×3 Matrix{Foo}:
    Foo()  #undef  #undef
 #undef    #undef  #undef
 #undef    #undef  #undef
 #undef    #undef  #undef
 #undef    #undef  #undef

julia> @undef! A[1]

julia> A
5×3 Matrix{Foo}:
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef
 #undef  #undef  #undef

julia> GC.gc()
bye world

Discussion

Undefs.jl is a meant as experimental convenience package and is not meant for production. The convenience constructor, undefs is meant to advance the conversation from recurring calls for a simpler way to allocate arrays without initialization. Additionally, the function undef! and the macro @undef! are highly experimental and use internal details of Julia’s array implementation to unassign elements from arrays of mutable structures.

While this package provides some convenient tools to play with, I encourage users to seek alternatives as described in the package README.

4 Likes

There’s a Base function for unsetting Array elements, if you don’t want to worry about the internal details yourself (which a prone to changing often). Undefing any other field by unsetting them is undefined behavior, and may lead to miscompilation or program crashes (pun intended)

1 Like

This package is not meant to be taken too seriously. What’s the name of the Base function?

Do you mean the exported C function jl_arrayunset ?

Yes, but I think there is also a faster version of that C function in Julia

Base._unsetindex! ?

Yes

1 Like

Is there a version intended for mutable structs or Base.RefValue?

No, the compiler relies on knowing it does not happen

1 Like

I see. So array elements can be unset, but not elements of mutable structs like Base.RefValue?

correct

I encountered a similar problem. From this very interesting discussion, I understand that it is only safe to unset entries in Array (and Memory), not in other structures. I also found jl_arrayunset and Base._unsetindex! but this latter function is neither documented nor public which seems a bit odd as it is used by some Julia packages.

BTW, to solve my problem, I have written a small package, called UnsetIndex.jl, to unset array entries and which has some overlap with Undefs.

UnsetIndex exports two symbols:

  • a function unsetindex! which calls Base._indexunset! if it exists (Julia version ≥ 1.3) and otherwise calls the C function jl_arrayunset;
  • a constant unset that is a singleton of a special type for which Base.setindex! and other base methods are extended to achieve similar results as the @undef! macro and more but with a more natural syntax.

To unset an entry of array A at index i, do one of:

unsetindex!(A, i)
A[i] = unset
setindex!(A, unset, i)

The entry is left untouched if it has a bit type. In any case, bound checking is
performed unless @inbounds is active. The only restriction is that A must be of type
Array or Memory.

Dot notation is supported. For example:

julia> using UnsetIndex

julia> A = split("This is a wonderful world indeed!")
6-element Vector{SubString{String}}:
 "This"
 "is"
 "a"
 "wonderful"
 "world"
 "indeed!"

julia> A[end] = unset
#undef

julia> A
6-element Vector{SubString{String}}:
    "This"
    "is"
    "a"
    "wonderful"
    "world"
 #undef

julia> A[1:2:5] .= unset
2-element view(::Vector{SubString{String}}, 1:2:3) with eltype SubString{String}:
 #undef
 #undef
 #undef

julia> A
6-element Vector{SubString{String}}:
 #undef
    "is"
 #undef
    "wonderful"
 #undef
 #undef

julia> @. A = unset
6-element Vector{SubString{String}}:
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef

For the last example, A[:] .= unset works as well and I have checked that these two are as fast as explictly writing a loop over all the elements of A.

1 Like

There is a feature request:

1 Like