Package extensions - structs inside extensions

Tooling Package Management
1

How can one make a struct defined in a package extension available to the user once the extension is loaded?

Suppose we have:

module MainFoo

using SomeLightDeps 

lightweight_func1 = ... 
lightweight_func2 = ... 

end

and its extension that should only be loaded if some heavy dependencies are installed:

module FooDataExt 

using MainFoo, HeavyDeps 

function extract_prop(x::HeavyType)
.... 
end

struct OneData 
someVar 
end

function OneData(x::HeavyType)
 someTemp1 = extract_prop(x)
 someTemp2 = lightweight_func1(someTemp1)
 someVar = HeavyDeps.heavy_func(someTemp2) 
end

struct TwoData
anotherVar :: AnotherHeavyType
end

function TwoData(yy::HeavyType)
 ....
end

end

I am able to make MainFoo precompile and usable. Next I load up the heavier dependencies and Julia dutifully loads and precompiles FooDataExt. So far so good … now how do I make use of OneData and TwoData in my program after the extension has been loaded? Both these structs and constructors are only defined in the extension module.

1 Like
2

If I’m remembering correctly from what @kristoffer.carlsson said on Slack (now lost), you should be able to writein your extension module

const MainFoo.OneData = OneData
const MainFoo.TwoData = TwoData

I’m not able to test it right now but I thnik that was what would work.

note that it’s not really recommended that you do this. If possible, you should just define OneData and TwoData in MainFoo.

1 Like
3

Wouldn’t we need to say export OneData and export TwoData somewhere to make them visible to the user after they issue using MainFoo ? What I found is that exporting from the extension module doesn’t do anything and to export from MainFoo … well the structs need to be defined in MainFoo.

4

Did you try? There’s nothing about export that need the exported variable to exist. You can just export a name and then only assign a value to that name when the extension is loaded

But I’ll also just repeat that I think what you’re asking to do is likely a bad idea. Why would you ever want the list of defined exported variables to change depending on where some package anywhere in the environment is loaded?

5

I did; and Julia can see the names because of the export in the main module but, it comes back undefined even though the extension is loaded and precompiled (as reported by the Package manager).

In [8]: using Cyclicity

In [9]: Cyclicity.

CyData            ElData            Scan              areaval
cumul_area        cycdiff           eval              include
make_lead_matrix  match_ends        mean_center       minmax_pairs
quad_norm         std_norm          totvar_norm
In [9]: Cyclicity.ElData
ERROR: UndefVarError: `ElData` not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:31
 [2] top-level scope
   @ REPL[9]:1
 [3] top-level scope
   @ /Applications/Julia-1.9.app/Contents/Resources/julia/share/julia/stdlib/v1.9/REPL/src/REPL.jl:1416

In [10]: Cyclicity.CyData
ERROR: UndefVarError: `CyData` not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:31
 [2] top-level scope
   @ REPL[10]:1
 [3] top-level scope
   @ /Applications/Julia-1.9.app/Contents/Resources/julia/share/julia/stdlib/v1.9/REPL/src/REPL.jl:1416

In line 9 for example I am doing tab completion and you can see the two data types are there but only vacuously.

My use case is a bit hard to explain but TL;DR is basically the main module has a bunch of functions that have solid day-to-day use. I want that Cyclicity package to install in a jiffy and be all-around useful. However, there are some analysis steps that are in some sense natural extensions or deeper next steps on the results of the functions listed above; but for this … I have to make use of some nonlinear constrained optimization (JuMP and Iopt) packages. So … I only want those helper structs & functions to be loaded up if the user consciously installs these packages.

Maybe I am thinking of it wrong … not quite sure what right way to approach this in Julia might be.

6

I think the most direct way is through get_extension:

Base.get_extension(YourPackage, :AnotherPkgExt).YourType

If YourType is used a lot, user code can do

using YourPackage
using AnotherPkg

const YourType = Base.get_extension(YourPackage, :AnotherPkgExt).YourType

It’s not too convenient, but this is official documented functionality.
When possible, it’s best to provide extension interface through methods to existing functions though.

3 Likes
7

we do need a more straight way to use functions and structures only defined in extension.