# Macro to "steal" types from another package

**URL:** <https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585>\
**Category:** General Usage\
**Created:** [May 6, 2022, 3:35am UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585 "2022-05-06T03:35:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [May 6, 2022, 3:35am UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/1 "2022-05-06T03:35:52Z")

</div>

Package A defines a bunch of types whose interface I would like to extend in Package B. However, to do this in a straightforward way, Package B requires those types to subtype `B.SomeAbstractType`. To mitigate this I am planning to automatically generate a bunch of new types in Package B - one for each type in Package A, which wraps each of the types in Package A and passes through their properties (that is, `getproperty(object_in_B::TypeInB, name) = getproperty(getfield(object_in_B, :atomic_object), name)`. Similarly, I will “forward” the methods in Package A that I need access to in Package B.

I’m supposing macros already exist to do this kind of thing and I wanting someone to point me in the right direction.

**Extra challenge.** In my use case, I actually want the wrapper types to be _mutable_ but the atomic types are _immutatable_. So a bonus, if that can also be taken care of.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [May 6, 2022, 7:51pm UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/2 "2022-05-06T19:51:04Z")

</div>

> [@ablaom](#):
>
> Package A defines a bunch of types whose interface I would like to extend in Package B. However, to do this in a straightforward way, Package B requires those types to subtype `B.SomeAbstractType` .

Since nobody answered in 16h, I’ll give my 2c…

This part may be easier to address by defining a union type in `B` that lists all the types you are interested in `A`, instead of trying to put them under the umbrella of a new abstract type:

```julia
julia> module A
           export A1, A2
           struct A1 end
           struct A2 end
       end
Main.A

julia> module B
           using ..A
           ATypes = Union{A1,A2}
           f(x::ATypes) = 1
           g(x::ATypes) = 2
       end
Main.B

```

But, probably this does not solve all your issues with dispatch. But anyway, that’s an idea. In the worst case it will bump your question 🙂

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [May 9, 2022, 9:12pm UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/3 "2022-05-09T21:12:06Z")

</div>

@lmiq Thanks for chiming in. Yeah, that’s not really what I want.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [May 9, 2022, 9:47pm UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/4 "2022-05-09T21:47:14Z")

</div>

@ablaom I guess you are talking about the TableTransforms.jl types as the immutable types and the wrapper types in MLJ.jl as the mutable types as discussed in [Hyperparameter tuning · Issue #67 · JuliaML/TableTransforms.jl · GitHub](https://github.com/JuliaML/TableTransforms.jl/issues/67)?

Why not work together in TableTransforms.jl to make these features more self-contained instead of introducing a “hacky” solution that involves stealing these types?

At the end of the day I understood that all you need is a mechanism to retrieve parameters and update them in structs in a hyperparameter tuning context?

We can certainly include hyperparameter tuning features in ways that are agnostic to specific ML frameworks.

cc: @eliascarv

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [May 9, 2022, 11:52pm UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/5 "2022-05-09T23:52:29Z")

</div>

Yes, @juliohm that is my use case. The problem is that currently models in MLJ must subtype `MLJModel` (and transformers subtype either `Unsupervised` or `Static`). I would like to change this requirement (abandon type heirarchy in MLJ in favour of pure traits) but I estimate a considerable disruption and some of my colleagues are not enthusiastic about this. So I don’t this happening soon. (By the way, it looks to me like transformers in TableTransforms.jl also have a local abstract type. Is subtyping this a hard requirement? Perhaps you can say more about that?)

One alternative would be for the `TableTransform` objects to subtype from the current MLJModelInteface.jl abstract types. I assumed you (quite reasonably) were trying to avoid that.

The solution I have in mind was used to “steal” LossFunction.jl types into MLJ. It seems to have worked out in that case but I agree this not ideal. Perhaps we should just keep integration on the backburner until both MLJ and TableTransforms mature further?

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [May 10, 2022, 1:08am UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/6 "2022-05-10T01:08:25Z")

</div>

> [@ablaom](#):
>
> The problem is that currently models in MLJ must subtype `MLJModel` (and transformers subtype either `Unsupervised` or `Static` ). I would like to change this requirement (abandon type heirarchy in MLJ in favour of pure traits) but I estimate a considerable disruption and some of my colleagues are not enthusiastic about this. So I don’t this happening soon.

That is precisely it @ablaom, I think we came across this issue before in the MLJ.jl issue tracker. There we discussed how difficult it is to work generically with models using a type hierarchy when the language doesn’t support multiple inheritance. I also wish we had a trait-based ML framework where we could mark a single model with multiple traits instead of forcing a single parent type. Hopefully, this will be sorted out in the future, even if it is a breaking change. This is a serious limitation of the MLJModelInterface.jl that could be brainstormed and addressed in a separate experimental package.

> [@ablaom](#):
>
> (By the way, it looks to me like transformers in TableTransforms.jl also have a local abstract type. Is subtyping this a hard requirement? Perhaps you can say more about that?)

In TableTransforms.jl, we introduced a set of abstract types in order to conveniently write fallback methods (e.g. `Transform`, `Stateless`, `Colwise`). They are not necessary in the sense that 3rd-party packages only need to implement the exposed API (a set of functions) for their own transform types (not tested). In other projects, however, I could not escape from a parent type in Julia because the fallback code was quite complex to reimplement (e.g. Meshes.jl).

> [@ablaom](#):
>
> One alternative would be for the `TableTransform` objects to subtype from the current MLJModelInteface.jl abstract types. I assumed you (quite reasonably) were trying to avoid that.

Yes, like with LossFunctions.jl, we are trying to write a self-contained package for tabular transforms that is agnostic to specific frameworks. I believe that there is great value in implementing generic modules in Julia that compose well instead of trying to solve everything in an umbrella package. For instance, LossFunctions.jl and TableTransforms.jl are already being used in GeoStats.jl, which is an umbrella package for “Geoscientific ML”. I could have implemented these features inside our umbrella instead, but that would inhibit other umbrellas from benefiting from the work. Also, keep in mind that first-time users can contribute much more easily to self-contained packages as opposed to huge umbrella projects. Given our community size, this is super relevant.

> [@ablaom](#):
>
> The solution I have in mind was used to “steal” LossFunction.jl types into MLJ. It seems to have worked out in that case but I agree this not ideal. Perhaps we should just keep integration on the backburner until both MLJ and TableTransforms mature further?

I am happy that you are interested in these efforts, and so feel free to proceed with the planned integration if you feel that is the best use of your time right now. My impression is that our community is so small that the time you will spend trying to “wrap” these modules in MLJ-specific types could instead be used to polish a really nice hyperparameter tuning interface for the TableTransforms.jl types that is agnostic to MLJ.jl, GeoStats.jl or any umbrella project, and yet meets all the requirements of real-world applications.

---

<div class="post-metadata">

**Author:** ![ablaom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ablaom/32/4889_2.png) [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Post date:** [May 16, 2022, 1:07am UTC](https://discourse.julialang.org/t/macro-to-steal-types-from-another-package/80585/7 "2022-05-16T01:07:15Z")

</div>

Thanks @juliohm for the detailed and thoughtful responses. I will take your advice and not attempt an interim but inferior integration at this time.