Is there a function to make an abstractly-typed variable "more" concrete?

General Usage Performance
1

Is there a way to find the most concrete type representation of a variable and apply it ?

For example

say we got our hands to a Dict{Any, Any}

julia> d = Dict{Any,Any}("one" => 1, "two" => 2)
Dict{Any, Any} with 2 entries:
  "two" => 2
  "one" => 1

it would be great if we could do

julia> conretisize(d)
Dict{String, Int64} with 2 entries:
  "two" => 2
  "one" => 1

and get back the most accurate typed version.

Similarly, Number[1, 2.0, 3] could be transformed to Float64[] and maybe Vector{Any}([1,missing, 2]) could be transformed to something like Vector{Union{Missing, Int64}}.

Remarks

Ofc, one could customize such an operation w.r.t. the situation. E.g., above we could get the type stable representation by doing Dict(k => v for (k,v) in d) for Dicts and [v for v in vector] for Vectors.
But I am wondering whether a more general implementation can be provided, that would work for all types.
If generalizing is hard, do you think it’s worth to provide it as an interface that could/should be implemented by the developers for newly defined structs ?

It’s obvious that conretisize cannot always return concrete type, so maybe the naming is a bit misleading.

As I understand it, this operation cannot be done in place as the memory representation changes.

I think this could be very useful to call in the middle of some functions to kill some type instabilities, if used together with the function barrier pattern.
(some more context in zulip mentioned as trytoinfer, but this post can be treated independently.)

2

Seems like a good idea. If you create a package, perhaps it should build upon ConstructionBase:

3

I would love to have such a function. Parsing Dict{Symbol, Any} is a Any’s plague, even when we know for sure the type of value.

4

As a start, would a generic function work for you?

# narrowtype(x) = x  # define if it shall not fail
narrowtype(x::Dict) = Dict(k => v for (k, v) in d)
narrowtype(x::AbstractArray) = map(identity, x)
# ...
1 Like
5

In some cases, broadcasting identity is all that you need:

julia> x = Any[1, 2, 3];

julia> typeof(identity.(x))
Vector{Int64} (alias for Array{Int64, 1})

julia> y = Any[1, 2, 3.0];

julia> typeof(identity.(y))
Vector{Real} (alias for Array{Real, 1})
4 Likes
6

“Shallow” narrowing is easy as shown here, while “recursive” is nontrivial but would also be nice.

4 Likes
7

Work on this might be coordinated with this package that just got registered:

Edit: @emmt

8

The promise is to be memory efficient, i.e. type stable. I noticed there are cases like that of Guillaume’s

julia> typeof(identity.(Any[1, 2, 3.0]))
Vector{Real} (alias for Array{Real, 1})

that do narrow the type but still are type instable.
I think in such cases some promotion needs to take place, e.g.:

julia> typeof(collect(promote(Any[1, 2, 3.0]...)))
Vector{Float64} (alias for Array{Float64, 1})
9

I think you’re using that phrase wrong.

This is a completely different goal than above. Sometimes one is useful, sometimes the other.

10

You mostly certainly will need to allocate a new data structure for the narrowed container, so the memory usage from type-unstability does not seem like a big concern here.

But if you want type-stability the only way is for the function to take an argument for which its type guarantees the type of the output, like a convert that only takes concrete types.

11

@nsajko

I think you’re using that phrase wrong.

Yes that was misleading, I was referring to memory efficiency as having the appropriate representation to make efficient computations.

This is a completely different goal than above.

The goal is solely to try to get concrete types. For this, both type-narrowing and promotion could be useful as demonstrated.

@Henrique_Becker

But if you want type-stability the only way is for the function to take an argument for which its type guarantees the type of the output, like a convert that only takes concrete types.

And the guaranteed type must also be concrete right ? Just want to make sure everybody is on the same page.

1 Like
12

Yes. But note that a type like Vector{Real} is concrete, it is accessing its elements that may cause type-instability.

2 Likes