`using` a package that uses `OrderedCollections` can silently add function methods?

[tomohiro_soejima]

(This might not be specific to Revise, but I have not looked for other examples yet. Will change the title if this topic applies more generally.)

sort usually does not work with dictionaries, but it starts to work after using Revise, even though Revise does not export sort.

test_dict = Dict(1=>2, 2=>1)
sort(test_dict) # this errors
using Revise
sort(test_dict) # this does not error and prints the following

OrderedCollections.OrderedDict{Int64, Int64} with 2 entries:
  1 => 2
  2 => 1

@which tells us the function is from OrderedCollections, even though OrderedCollections itself is brought in scope.

julia> @which sort(test_dict)
sort(d::Dict; args...) in OrderedCollections at deprecated.jl:70

julia> OrderedCollections
ERROR: UndefVarError: OrderedCollections not defined

julia> Revise.OrderedCollections # this is in scope, and it gets printed without a qualifier
# which might be why @which says sort is from OrderedCollections
OrderedCollections

This seems to have something to do with the fact Revise.jl uses OrderedCollections, but I don’t quite understand how it works. Specifically,

1. I thought using only brings in exported names and the module name into scope. Why does the behavior of sort change?
2. Does this mean I always need to worry about the possibility that an imported package might silently add some methods to functions?

Thanks in advance!

[rdeits]

This is a classic case of type piracy, in this case within OrderedCollections. Revise isn’t actually relevant here except that it happens to be the first thing in your environment to use OrderedCollections and thus trigger the issue. This was actually already identified as a bug in OrderedCollections: Type piracy? · Issue #25 · JuliaCollections/OrderedCollections.jl · GitHub and was fixed by deprecating that method: do not support sorting a Dict by dpo · Pull Request #26 · JuliaCollections/OrderedCollections.jl · GitHub (which will hopefully be removed completely soon).

@tomohiro_soejima

Does this mean I always need to worry about the possibility that an imported package might silently add some methods to functions?

Only if that package commits type piracy, which is exactly why we always tell people to avoid that.

[tomohiro_soejima]

Thanks for your quick response! That makes sense.

As a follow-up question, does this mean there is nothing users can do to protect ourselves from type piracy, other than to hope they don’t exist? I initially thought modules that are called inside a package wouldn’t be an issue because it’s separated by two usings (I call Revise which calls OrderedCollections), but I suppose that is not the case? In other words, not only do I need to hope a package I load is not committing type piracy, but I also need to hope all dependencies of the package did not commit type piracy?

[rdeits]

@tomohiro_soejima

I initially thought modules that are called inside a package wouldn’t be an issue because it’s separated by two using s (I call Revise which calls OrderedCollections ), but I suppose that is not the case?

No, that has no effect. There is only ever one instance of a given module, so the OrderedCollections inside Revise is the same one you get everywhere else (it is possible to have two unrelated modules with the same name, but that’s unrelated to what’s going on here).

@tomohiro_soejima

As a follow-up question, does this mean there is nothing users can do to protect ourselves from type piracy, other than to hope they don’t exist?

You might try Pirate Hunter (I haven’t used it myself)

@tomohiro_soejima

In other words, not only do I need to hope a package I load is not committing type piracy, but I also need to hope all dependencies of the package did not commit type piracy?

Basically yes. In my experience, this is rarely a problem, but it is something to be aware of.

[tomohiro_soejima]

Great, thanks for the clarifications!