Need help with resolving method ambiguities

[tisztamo]

Just started to experiment with implementing a proxy type, but I failed very early:

module Proxies

export AbstractProxy, Proxy

abstract type AbstractProxy end

struct Proxy{TInner} <: AbstractProxy
    inner::TInner
    Proxy{TInner}(inner) where TInner = new{TInner}(inner)
    Proxy(inner) = begin
        p = Proxy{typeof(inner)}(inner)
        return p
    end
end


inner(p) = p.inner

Base.convert(targettype::Type{<:T}, p::Proxy{TInner}) where {T, TInner} = begin
    return convert(targettype, inner(p))
end

end # module

using Proxies
using Test

@testset "Proxy creation" begin
    p = Proxy(42)
    @test p isa Proxy
    @test Proxies.inner(p) == 42
end

@testset "transparent proxing" begin
    @test convert(Int, Proxy(42)) isa Int
    @test convert(Int, Proxy(42.0)) isa Int
    @test Proxy(42) == 42
    @test_broken Proxy(42) + 42 == 84
end

Test Summary:  | Pass  Total
Proxy creation |    2      2
Test Summary:       | Pass  Total
transparent proxing |    2      2
ERROR: LoadError: MethodError: convert(::Type{Any}, ::Proxy{Int64}) is ambiguous. Candidates:
  convert(::Type{Any}, x) in Base at essentials.jl:170
  convert(targettype::Type{var"#s13"} where var"#s13"<:T, p::Proxy{TInner}) where {T, TInner} in Proxies at /home/krisztian/projects/Proxies.jl/src/Proxies.jl:19
Possible fix, define
  convert(::Type{Any}, ::Proxy{TInner}) where TInner
Stacktrace:
 [1] Base.RefValue{Any}(::Proxy{Int64}) at ./refvalue.jl:8
 [2] Ref{Any}(::Proxy{Int64}) at ./refpointer.jl:96
 [3] inferencebarrier(::Any) at ./essentials.jl:718
 [4] show_default at ./show.jl:389 [inlined]
 [5] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:384
 [6] show_delim_array(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Tuple{DataType,Proxy{Int64}}, ::Char, ::Char, ::Char, ::Bool, ::Int64, ::Int64) at ./show.jl:776
 [7] show_delim_array at ./show.jl:761 [inlined]
 [8] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Tuple{DataType,Proxy{Int64}}) at ./show.jl:794
 [9] _show_default(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:406
 [10] show_default at ./show.jl:389 [inlined]
 [11] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:384
 [12] sprint(::Function, ::MethodError; context::Pair{Symbol,Bool}, sizehint::Int64) at ./strings/io.jl:103
 [13] Test.Error(::Any, ::Any, ::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:162
 [14] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1121
 [15] top-level scope at /home/krisztian/projects/Proxies.jl/test/runtests.jl:11
 [16] include(::String) at ./client.jl:457
 [17] top-level scope at none:6
in expression starting at /home/krisztian/projects/Proxies.jl/test/runtests.jl:10
caused by [exception 1]
MethodError: convert(::Type{Any}, ::Proxy{Int64}) is ambiguous. Candidates:
  convert(::Type{Any}, x) in Base at essentials.jl:170
  convert(targettype::Type{var"#s13"} where var"#s13"<:T, p::Proxy{TInner}) where {T, TInner} in Proxies at /home/krisztian/projects/Proxies.jl/src/Proxies.jl:19
Possible fix, define
  convert(::Type{Any}, ::Proxy{TInner}) where TInner
Stacktrace:
 [1] Base.RefValue{Any}(::Proxy{Int64}) at ./refvalue.jl:8
 [2] Ref{Any}(::Proxy{Int64}) at ./refpointer.jl:96
 [3] inferencebarrier(::Any) at ./essentials.jl:718
 [4] show_default at ./show.jl:389 [inlined]
 [5] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:384
 [6] show_delim_array(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Tuple{DataType,Proxy{Int64}}, ::Char, ::Char, ::Char, ::Bool, ::Int64, ::Int64) at ./show.jl:776
 [7] show_delim_array at ./show.jl:761 [inlined]
 [8] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Tuple{DataType,Proxy{Int64}}) at ./show.jl:794
 [9] _show_default(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:406
 [10] show_default at ./show.jl:389 [inlined]
 [11] show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any) at ./show.jl:384
 [12] sprint(::Function, ::MethodError; context::Pair{Symbol,Bool}, sizehint::Int64) at ./strings/io.jl:103
 [13] Test.Error(::Any, ::Any, ::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:162
 [14] do_test(::Test.ExecutionResult, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:518
 [15] top-level scope at /home/krisztian/projects/Proxies.jl/test/runtests.jl:13
 [16] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
 [17] top-level scope at /home/krisztian/projects/Proxies.jl/test/runtests.jl:11
 [18] include(::String) at ./client.jl:457
 [19] top-level scope at none:6
ERROR: Package Proxies errored during testing

I would really appreciate any help on this!

[johnmyleswhite]

In your original code, I get the following error:

julia> p = Proxy(42)
Error showing value of type Proxy{Int64}:
ERROR: MethodError: convert(::Type{Any}, ::Proxy{Int64}) is ambiguous. Candidates:
  convert(::Type{Any}, x) in Base at essentials.jl:170
  convert(targettype::Type{var"#s2"} where var"#s2"<:T, p::Proxy{TInner}) where {T, TInner} in Main at REPL[4]:1
Possible fix, define
  convert(::Type{Any}, ::Proxy{TInner}) where TInner

The last line suggests an ambiguity fix:

function Base.convert(::Type{Any}, ::Proxy{TInner}) where TInner
    convert(Any, inner(p))
end

That gets rid of the ambiguities, but then you have more methods to implement to get your tests passing.

[tisztamo]

Thank you, not clear how I have missed that!

[kristoffer.carlsson]

Just as a note, it is quite questionable to have a type that converts to absolutely any other type. Conversions should usually result in something that is very similar to the original thing (1 → 1.0) and it is unclear if you can have a type such that this is true for all the types it wraps.

It might be better to have a designated unwrap method or something along those lines.

[tisztamo]

Thanks for noting this! My understanding of conversion and promotion is not perfect yet, so possibly even the goal of this method could be improved. With

Base.convert(targettype::Type{<:T}, p::Proxy{TInner}) where {T, TInner} = begin
    return convert(targettype, inner(p))
end

I have tried to say in Julia that “If a conversion is available from TInner to T, then here I provide one from Proxy{TInner} to T”

[tomerarnon]

On the topic of

Finding and fixing invalidations: now, everyone can help reduce time-to-first-plot

I just posted a new blog post on recent work reducing Julia’s latency, a.k.a., “time to first plot” and even “time to second plot (after loading more code)”. The blog post describes recent work on diagnosing and eliminating invalidations, events that cause Julia to have to recompile previously-compiled code. The blog post focuses on (1) explaining the underlying ideas and (2) briefly summarizing the overall progress we’ve made so far. I’m writing this post to emphasize to package developers the…

and

A playful competition: who can define a method that invalidates the most code?

Yikes, realized I forgot to announce the results on Friday. To evaluate the submissions, I tested each using a script that I executed from the linux prompt to ensure there weren’t differences in my typing at the REPL (since hitting certain keys can force the compilation of new methods). In a couple of cases, this led to a pretty different count than provided by the submitter, perhaps mostly because some REPL methods did not compile. (At least the rank order was consistent with the numbers provid…

The method:

function Base.convert(targettype::Type{<:T}, p::Proxy{TInner}) where {T, TInner}
    return convert(targettype, inner(p))
end

causes 1934 invalidations…

[tisztamo]

Great, better than the winning convert in the non-pirating category!

I have thought that implementing a proxy type (similar to Proxy - JavaScript | MDN ) will be relatively simple, so a good way to learn more about how types are working. Never thought that it will be performant when implemented naively, because a lot of evaling seems to be needed. (Not sure if non-naive implementation is possible though)

[oxinabox]

My instinct is that you can’t prevent those and you don’t want to.
I am pretty sure it would require changes to how the things you invalidated were written.
Or to the code that calls them.

Invalidations are not, as a rule, a problem for package authors to worry about.
It’s a problem the compiler and standard library folk to worry about.

[tomerarnon]

Reading through that link, I think a naive implementation (even a not very naive one…) is not possible in julia. You would need a way to replace arbitrary function calls on the type with different ones (at runtime, without macros and so on presumably, or else you severely limit the usability of Proxy). The way javascript is interpretted is different enough from how julia is compiled that there can’t be an exact equivalent to Proxy without call overloading on ::Function or ::Any

With a macro you could do:

@proxy p let getproperty = blah, setproperty! = blah

    p.x = p.x + p.y # can be translated by the macro to whatever

end

but doesn’t really fill the same niche as Proxy. Normally you would just define a new type or a new method, depending on what behavior you need changed…