How to correctly write package extension code?

General Usage
1

For example, I have a main package called NeuralNetworks, and an extension called MNIST. Now I want to load MNIST if package MLDatasets.jl is already loaded.

I wrote the following code in NeuralNetworks:

function loaddata end

if !isdefined(Base, :get_extension)
    include("../ext/MNIST.jl")
end

where function loaddata is what I want to extend in the extension.

And I wrote the following code in MNIST:

module MNIST

using MLDatasets: MLDatasets
if isdefined(Base, :get_extension)
    using NeuralNetworks
    import NeuralNetworks: loaddata
else
    using ..NeuralNetworks
    import ..NeuralNetworks: loaddata
end

export loaddata

function loaddata(type=:train)
    ...
end

end

My Project.toml looks like:

[weakdeps]
MLDatasets = "eb30cadb-4394-5ae3-aed4-317e484a6458"

[extensions]
MNIST = "MLDatasets"

Now I try to use the code:

In [1]: using NeuralNetworks: loaddata

In [2]: methods(loaddata)
# 0 methods for generic function "loaddata" from NeuralNetworks

In [3]: using MLDatasets

In [4]: methods(loaddata)
# 0 methods for generic function "loaddata" from NeuralNetworks

As you see, even if MLDatasets is loaded, loaddata is still not extended. What is wrong with my code? Thanks!

2

I’ve never got a package extension to load correctly when developing inside VSCode (would love to see if anyone has a fix for this!) - is that what you’re using for this? When I run your example in a terminal, it works for me:

julia> using MLDatasets

julia> methods(loaddata)
# 2 methods for generic function "loaddata" from NeuralNetworks:
 [1] loaddata()
     @ MNIST C:\Users\User\.julia\packages\NeuralNetworks\ZPIX0\ext\MNIST.jl:16
 [2] loaddata(type)
     @ MNIST C:\Users\User\.julia\packages\NeuralNetworks\ZPIX0\ext\MNIST.jl:16
1 Like
3

Hi Daniel,
I still cannot make it work. Both in VSCode and in a separate terminal. I wonder how you did this. Were you on commit 424c8f4?

In [1]: using NeuralNetworks

In [2]: using NeuralNetworks: loaddata

In [3]: methods(loaddata)
# 0 methods for generic function "loaddata" from NeuralNetworks

In [4]: using MLDatasets

In [5]: methods(loaddata)
# 0 methods for generic function "loaddata" from NeuralNetworks
4

Here’s my complete session:

C:\Users\User>julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.0 (2023-05-07)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.9) pkg> activate --temp
  Activating new project at `C:\Users\User\AppData\Local\Temp\jl_QHuyqM`

(jl_QHuyqM) pkg> add https://github.com/singularitti/NeuralNetworks.jl, MLDatasets

julia> using NeuralNetworks

julia> using NeuralNetworks: loaddata

julia> methods(loaddata)
# 0 methods for generic function "loaddata" from NeuralNetworks

julia> using MLDatasets

julia> methods(loaddata)
# 2 methods for generic function "loaddata" from NeuralNetworks:
 [1] loaddata()
     @ MNIST C:\Users\User\.julia\packages\NeuralNetworks\ZPIX0\ext\MNIST.jl:16
 [2] loaddata(type)
     @ MNIST C:\Users\User\.julia\packages\NeuralNetworks\ZPIX0\ext\MNIST.jl:16
5

I get it. I did not install MLDatasets together alongside NeuralNetworks. MLDatasets is installed in my root path and NeuralNetworks is a local project. It seems that the new package has to be under the same environment as the main package. It kind of limits the usefulness of this functionality.

Thank you!

6

I also have frequent troubles with extensions not loading in VSCode, I thought it might be due to Revise. Are you saying this works fine 100% of the time in the standard REPL?

7

I should’ve said command prompt rather than terminal. Anywhere from inside VSCode never works for me when developing a package, but 100% of the time outside of VSCode.