I am preparing a new package to handle periodic matrices, which has two package extensions, say PackageExt1 and PackageExt2, which additionally define and manipulate two new (periodic) matrix types PM1 and PM2, respectively. PackageExt1 is loaded if the Symbolics package is installed, while PackageExt2 is loaded when the ApproxFun package is installed. I would like to provide within PackageExt1 (or PackageExt2) functions which performs conversions between matrices of types PM1 and PM2. How it is possible to define such conversions functions, which should work only if both PackageExt1 and PackageExt2 are loaded? Perhaps a conditional compilation of codes is feasible, depending on which packages are installed ? Or is there a possibility to define an extension which is loaded if both packages Symbolics and ApproxFun are installed?
You can write a PackageExt1Ext2 that is loaded when both are loaded and put the common part is loaded.
See line 38 how to specify that (the name upfront should maybe reflect that better than here)
Addon – for the backward compatibility with requires you have to nest that of course
I implemented your suggestion, but later I realized that the extensions have namespaces which can not be simply seen from outside, and in my case, I even would need that the defined types PM1 and PM2 can be seen from the third extension performing the conversions between these types. I read some discussions on exporting names from extensions and I decided to not use this techniques, although I was hopping to significantly reduce the burden of loading all packages all the time using a selective loading approach. I still hope that a clever solution will be found for this issue in the future. For me it would be sufficient a kind of conditional compilation of codes in the extensions, without being encapsulated into modules.
My extensions until now never defined any types, they only extend existing functions with further methods (dispatches).
The reason is, that this is how extensions are meant, they are not meant to define new types (unless locally within that extension useful).
So that is the conditional compilation you refer to. The idea is that all types already exist, even the functions all already exist. You just “extend functionality” with special dispatches in the extensions. That is at least the design idea.
Do you have a concrete example where you really need a “local” type within an extension?
My package PeriodicSystems (in development) relies on several types of periodic matrices, of which the PeriodicSymbolicMatrix type involves the use ot the Symbolics package and the FourierFunctionMatrix type involves the ApproxFun package. Other 8 types are also defined, 4 for continuous time dependence and 4 for discrete time dependence. The issue is that I can not define the symbolic type without loading the Symbolics package and the same is true for the other type. Since both Symbolics and ApproxFun packages have a lot of dependencies, I was pondering the possibility to load them only if they are actually needed. This is my concrete case.
I would like to mention shortly the context for my current work. Becasue of many dependencies, the testing times for the PeriodicSystems package started to be huge (I am ashamed to say that without using a more clever caching approach, one CI run took more that 5 hours, although on my local computer with Windows it run in one hour!). So I decided to restructure this package, by moving stuff in supporting packages, which can have independent usage. The first is a package to handle periodic matrices, the second to solve periodic matrix differential equations (Lyapunov, Riccati) and a third having the intended functionality for periodic dynamic systems. I just started with PeriodicMatrices, where I intended to use the extensions based approach to further reduce the loading burden. Sorry for so much details and thanks for your time.
Oh, that sounds challenging, without a very close look it is probably hard to propose something, at first glance I would have guessed that something PeriodicMatrix might help, because you currently define several times very similar things in PeriodicSystems.jl/src/types/PeriodicMatrices.jl at master · andreasvarga/PeriodicSystems.jl · GitHub that always have the form PeriodicXMatrix{Domain, T} you could rework that to PeriodicMatrix{X,Domain,T} and X could become the <:Num or Matrix{<:Num} in the symbolic case, F<:Function in the function case Array{T,3} in the array case (maybe reorder the parameters when X depends in T I think it has to be behind T) – and <:Vector{<:Matrix} for the “original” periodic matrix.
That for best of cases also reduces your amount of code, but it especially allows to define the symbol parameter part/case in an extension.
Yes this is a quick shot from a 3 minute look at your code, but maybe the hint is helpful.
Thank for your suggestions. I have to reflect what changes it would involve to redefine the periodic matrix structures. It seems that it could be worth even if no extensions are involved.
Here is the code intended for the PeriodicSymbolicMatrix type in the corresponding extension, together with some constructors. I wonder how these constructors would look if PeriodicMatrix{Domain,T,X} is used from the main module PeriodicMatrices.
"""
PeriodicSymbolicMatrix(F, T; nperiod = k) -> A::PeriodicSymbolicMatrix
Continuous-time periodic symbolic matrix representation.
The continuous-time periodic symbolic matrix object `A` is built from `F`, a
symbolic real matrix or vector of symbolic variable `t`,
the associated time period `T` and the associated number of subperiods
specified via the keyword argument `nperiod = k`.
It is assumed that `F(t) = F(t+T/k)` for any real time value `t`.
The symbolic matrix `F`, the period `T` and the number of subperiods `k`
can be accessed via `A.F`, `A.period` and `A.nperiod`, respectively.
"""
struct PeriodicSymbolicMatrix{Domain,T} <: AbstractPeriodicArray{Domain,T}
F::Matrix{<:Num}
period::Float64
nperiod::Int
end
# additional constructors
function PeriodicSymbolicMatrix{:c,T}(F::VecOrMat{T}, period::Real; nperiod::Int = 1) where {T <: Num}
period > 0 || error("period must be positive")
nperiod > 0 || error("number of subperiods must be positive")
# check that array F is depending only on t
tt = rand()
@variables t
Ft = substitute.(F, (Dict(t => tt),))
m, n = size(Ft,1), size(Ft,2)
any(length.(Symbolics.get_variables.(Ft)) .> 0 ) && error("t must be the only variable in F")
PeriodicSymbolicMatrix{:c,T}(n == 1 ? reshape(F,m,n) : F, Float64(period), nperiod)
end
PeriodicSymbolicMatrix(F::VecOrMat{T}, period::Real; nperiod::Int = 1) where {T <: Union{Num,Real}} =
PeriodicSymbolicMatrix{:c,Num}(Num.(F), period; nperiod)
function PeriodicSymbolicMatrix{:c,T}(A::PeriodicSymbolicMatrix, period::Real) where {T}
period > 0 || error("period must be positive")
Aperiod = A.period
r = rationalize(Aperiod/period)
n, d = numerator(r), denominator(r)
min(n,d) == 1 || error("new period is incommensurate with the old period")
if period >= Aperiod
PeriodicSymbolicMatrix{:c,T}(A.F, Aperiod*d, A.nperiod*d)
elseif period < Aperiod
nperiod = div(A.nperiod,n)
nperiod < 1 && error("new period is incommensurate with the old period")
PeriodicSymbolicMatrix{:c,T}(A.F, Aperiod/n, A.nperiod)
end
end
You could either keep the constructor with PeriodicSymbolicMatrix (and fill in X as Matrix{T} yourself, but even changing F::VecOrMat{T) to F::X should do the job on a PeriodicMatrix constructor; to not have a parametric constructor you could do
function PeriodicMatrix(F::X, period::Real, nperiod::Int = 1; domain=:c) where {X<:VecOrMat{T}, T<: Num}
# .... stuff as before
return PeriodicMatrix{domain, T, X}( ... ) #as before
end
then a user can really just use PeriodicMatrix(...) without parameters, but I did not check that with your rest in practice.
And I feel it would also reduce code amount in other places.
In PeriodicSystems the PeriodicSymbolicMatrix is used as a type. This feature would be lost with the new scheme I assume.
Well you could still use either generically PeriodicMatrix in there if you can allow for all periodic matrices or restrict the parameters
PeriodicSystems{P} where {P <: PeriodicMatrix{D, T, X} where {X <: ...}) (whatever your restrictions on X are.
OK. I will prepare a PeriodicMatrices package using the old definitions (to be sure it works within PeriodicSystems) and without using extensions. Then I will try the scheme proposed by you, only within PeriodicMatrices. In this moment, the changes appear to me being substantial! I will then come back to the extension issue if I will manage with all changes. Thanks for your support.
I do agree that this is a substantial change, but I hope it makes the code a bit cleaner and less redundant and allows for the extensions to work nicely 
I almost finished with the first step, but I used this time a new definition for the PeriodicSymbolicMatrix type (following your suggestion):
struct PeriodicSymbolicMatrix{Domain,T,X} <: AbstractPeriodicArray{Domain,T}
F::X
period::Float64
nperiod::Int
end
Could this solve my dilema to keep PeriodicSymbolicMatrix as a concrete type and make all constructors from an extension depending on Symbolics visible (i.e., callable as before)?
If this would work, the transition to the extension based version would be very straightforward.
I wonder if adding something like
PeriodicSymbolicMatrix() = nothing
would be still necessary (to have PeriodicSymbolicMatrix also as a function name)?
If you define this struct in the main package (not the extension) you should be fine. The constructors you could put into the extension.
Thanks. This was my intention (sorry not being clear enough). I still have a few functions defined in the extensions which I would like to also call from outside (e.g., for test purposes). For example, to access MyFunction, is the scheme
MyFunction() = nothing
defined in the main package, generally recommended?
Hm that might be misleading since you define the function with that. I would just attach a doc string to introduce in the main package
"""
MyFunction
...
"""
MyFunc(args...)
maybe with a note that this function works only if the extension is loaded.
I think defining empty functions like that isn’t too uncommon. I’ve seen it in a few packages. Note though that adding
MyFunction() = nothing
adds a method to MyFunction. You can just do
function MyFunction end
Just adding a docstring to the Symbol itself does work in that it shows the docstring, but it will show a warning alongside it:
julia> @doc "blah" NoSymbol
NoSymbol
help?> NoSymbol
search:
Couldn't find NoSymbol
Perhaps you meant Symbol or Number
blah
julia> function NoSymbol end
NoSymbol (generic function with 0 methods)
julia> @doc "blah" NoSymbol
NoSymbol
help?> NoSymbol
search: NoSymbol
blah
Oh then attach the doc string to function bla end – I remembered that wrongly.