# Adding Hidden Fields to Parametric Types

**URL:** <https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448>\
**Category:** General Usage\
**Created:** [August 24, 2020, 11:45am UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448 "2020-08-24T11:45:44Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![zekeriya.sari](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zekeriya.sari/32/13695_2.png) [@zekeriya.sari](https://discourse.julialang.org/u/zekeriya.sari)\
**Post date:** [August 24, 2020, 11:45am UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/1 "2020-08-24T11:45:44Z")

</div>

Say I want to define a macro which adds a hidden field to a user defined type. See the minimal working example below

```julia
module Foo 

""" Modify ex by pushing hidden field. """
function push_hidded_field!(ex)
    paramex = ex.args[2]
    bodyex = ex.args[3]
    push!(bodyex.args, :(y::S))
    push!(paramex.args, :S) 
    ex
end

macro deftype(ex)
    push_hidded_field!(ex)
    quote 
        $ex 
    end |> esc
end

export @deftype

end 

```

Now consider some of the use case of this macro.

```julia
using .Foo 

# Initial definiton is not a problem. 
@deftype struct Bar{T} 
    x::T 
end

```

There is nothing wrong in this case.

```julia
julia> dump(Bar) 
UnionAll
  var: TypeVar
    name: Symbol T
    lb: Union{}
    ub: Any
  body: UnionAll
    var: TypeVar
      name: Symbol S
      lb: Union{}
      ub: Any
    body: Bar{T,S} <: Any
      x::T
      y::S

julia> fieldnames(Bar) 
(:x, :y)

```

`Bar` was defined as expexted with a hidden fieldname `y`. The problem occurs when I try to parameterize `x` with `S`.

```julia
julia> @deftype struct Bar{S} 
           x::S 
       end
ERROR: syntax: function static parameter names not unique around /tmp/deleteme4.jl:17
Stacktrace:
 [1] top-level scope at REPL[7]:1
 [2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1088

```

The complaint is that the names are not unique. This means that the user calling the macro cannot parametrize its fieldname `x` with `S`. I think this is restrictive and needs a better approach.

To solve this problem, let us consider that we parameterize our hidden field name `y` with `gensym` so that we avoid duplicate names. So let us redefine the macro

```julia
module Foo 

""" Modify ex by pushing hidden field. """
function push_hidded_field!(ex)
    paramex = ex.args[2]
    bodyex = ex.args[3]
    S = gensym() # Hidden parameter type be different with those of the user. 
    push!(bodyex.args, :(y::$S))
    push!(paramex.args, S) 
    ex
end

""" Define the macro """
macro deftype(ex)
    push_hidded_field!(ex)
    quote 
        $ex 
    end |> esc
end

export @deftype

end 

```

and let us try some use cases.

```julia
julia> using .Foo 

julia> @deftype struct Bar{T} 
           x::T 
       end

julia> dump(Bar) 
UnionAll
  var: TypeVar
    name: Symbol T
    lb: Union{}
    ub: Any
  body: UnionAll
    var: TypeVar
      name: Symbol S
      lb: Union{}
      ub: Any
    body: Bar{T,S} <: Any
      x::T
      y::S

julia> fieldnames(Bar)
(:x, :y)

```

There is nothing wrong in the **first** call. The problem occurs when the macro is called multiple times in the **same julia session**.

```julia
julia> @deftype struct Bar{T} 
           x::T 
       end
ERROR: invalid redefinition of constant Bar
Stacktrace:
 [1] top-level scope at REPL[1]:16
 [2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1088

```

This time the problem is that after being defined `Bar` is constant and cannot be redefined.

Any suggestions about how to get rid of these restrictions?

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [August 24, 2020, 12:04pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/2 "2020-08-24T12:04:00Z")

</div>

> [@zekeriya.sari](#):
>
> This time the problem is that after being defined `Bar` is constant and cannot be redefined.

That’s exactly the problem: types can not be re-defined. This hasn’t anything to do with your code, but is rather a design choice of Julia (which has been discussed numerous times, especially in the context of `Revise`; in case you’re interested in the reasons behind this, searching for these keywords should lead you to relevant discourse threads)

Apart from the context of interactive development, during which you might want to update the definition of an existing type (which is the `Revise` scenario), is there a reason why you’d want to have two successive definitions of the same type in your code?

---

<div class="post-metadata">

**Author:** ![zekeriya.sari](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zekeriya.sari/32/13695_2.png) [@zekeriya.sari](https://discourse.julialang.org/u/zekeriya.sari)\
**Post date:** [August 24, 2020, 12:15pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/3 "2020-08-24T12:15:47Z")

</div>

> [@ffevotte](#):
>
> is there a reason why you’d want to have two successive definitions of the same type in your code?

Let us assume that the macro is defined such that the fieldnames except default values such as (as in the case of `Base.@kwdef`)

```julia
using .Foo 

@deftype struct Bar{T}
    x::T = t -> t + 1. 
end 

```

When this call is executed, `Bar` is defined. The default is called in the run time. Let us say I realize that I defined the default value of `x` wrong and want to correct it

```julia
@deftype struct Bar{T}
    x::T = t -> t + 10. 
end 

```

I cannot do this because I cannot redefine `Bar`

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 24, 2020, 12:24pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/4 "2020-08-24T12:24:05Z")

</div>

> [@zekeriya.sari](#):
>
> ```julia
> julia> fieldnames(Bar) 
> (:x, :y)
> 
> ```
> 
> `Bar` was defined as expexted with a hidden fieldname `y` .

I may be missing something, but I am not sure in what sense you consider `y` “hidden”, when in fact it is like any other field, just part of the type definition is generated by a macro.

If you just want to add the same set of fields to various types, I would suggest putting them in a `stuct` and composing.

---

<div class="post-metadata">

**Author:** ![zekeriya.sari](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zekeriya.sari/32/13695_2.png) [@zekeriya.sari](https://discourse.julialang.org/u/zekeriya.sari)\
**Post date:** [August 24, 2020, 12:28pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/5 "2020-08-24T12:28:40Z")

</div>

> [@Tamas\_Papp](#):
>
> I may be missing something, but I am not sure in what sense you consider `y` “hidden”, when in fact it is like any other field, just part of the type definition is generated by a macro.

There is nothing special about the word `hidden`, We may call it as `additional` or `extra`. I just want to add additional fieldnames to a type that the user does not provide.

> [@Tamas\_Papp](#):
>
> If you just want to add the same set of fields to various types, I would suggest putting them in a `stuct` and composing.

Of course, composing types may be a solution. But I wondered if there is a possible solution using macros.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [August 24, 2020, 1:16pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/6 "2020-08-24T13:16:35Z")

</div>

> [@zekeriya.sari](#):
>
> I cannot do this because I cannot redefine `Bar`

Yes you can:

```julia
julia> Base.@kwdef struct B{T}
       a::T = 1
       end

julia> B()
B{Int64}(1)

julia> Base.@kwdef struct B{T}
       a::T = 11
       end

julia> B()
B{Int64}(11)

```

because this does not re-define the type but only the keyword constructor method which fills in the `1` or `11`.

---

<div class="post-metadata">

**Author:** ![zekeriya.sari](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zekeriya.sari/32/13695_2.png) [@zekeriya.sari](https://discourse.julialang.org/u/zekeriya.sari)\
**Post date:** [August 24, 2020, 1:18pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/7 "2020-08-24T13:18:55Z")

</div>

What I meant was that I cannot do this using `@deftype` macro, not using `Base.@kwdef`.

And, if I change the symbol of the type I end up with the same error.

```julia
julia> Base.@kwdef struct B{S}
       a::S = 11
       end
ERROR: invalid redefinition of constant B
Stacktrace:
 [1] top-level scope at util.jl:459
 [2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1088

```

So I think `Base.@kwdef` also tries to redefine the type. At least what I see from this line is this. [https://github.com/JuliaLang/julia/blob/41781b1cb01fa8d2f387a1452de8e13b5295c08f/base/util.jl#L463](https://github.com/JuliaLang/julia/blob/41781b1cb01fa8d2f387a1452de8e13b5295c08f/base/util.jl#L463)

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [August 24, 2020, 1:43pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/8 "2020-08-24T13:43:00Z")

</div>

AFAIU the only way a type redefinition is allowed is if the new definition is exactly the same as the old one. This is the case with `Base.@kwdef` when only default values are changed (because default values are handled via specific constructors, not in the type definition).

But it is not the case with your macro as it’s currently written, since the name of the extra type parameter changes each time the macro expands (it gets gensymmed anew)

A possible workaround could be to always use the same gensymmed symbol for all types extended this way. I think this is legit because the only hygiene issue you want to avoid would be a clash between the extra type parameter name and any user-provided type parameter.

To expand on your initial example (I made it so `Foo.@deftype` implicitly forwards the type definition to `Base.@kwdef` so that default values can easily get defined):

```julia
module Foo
export @deftype

# Gensymmed only once, used for all type definitions
const EXTRA_FIELD_TYPE = gensym()

function push_hidded_field!(ex)
    paramex = ex.args[2]
    bodyex = ex.args[3]
    # Let's put a default value for the extra field as well
    push!(bodyex.args, :(y::$EXTRA_FIELD_TYPE = 0))
    push!(paramex.args, EXTRA_FIELD_TYPE)
    ex
end

macro deftype(ex)
    push_hidded_field!(ex)
    quote
        Base.@kwdef $ex
    end |> esc
end

end

```

This behaves how (I think) you’d expect:

```julia
julia> using .Foo

julia> @deftype struct Bar{T}
           x::T = 42
       end

julia> Bar()
Bar{Int64,Int64}(42, 0)

julia> @deftype struct Bar{T}
           x::T = 43.0
       end

julia> Bar()
Bar{Float64,Int64}(43.0, 0)

```

---

<div class="post-metadata">

**Author:** ![zekeriya.sari](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zekeriya.sari/32/13695_2.png) [@zekeriya.sari](https://discourse.julialang.org/u/zekeriya.sari)\
**Post date:** [August 24, 2020, 1:50pm UTC](https://discourse.julialang.org/t/adding-hidden-fields-to-parametric-types/45448/9 "2020-08-24T13:50:39Z")

</div>

> [@ffevotte](#):
>
> A possible workaround could be to always use the same gensymmed symbol for all types extended this way. I think this is legit because the only hygiene issue you want to avoid would be a clash between the extra type parameter name and any user-provided type parameter.

That was exactly what I was looking for. Thank your @ffevotte .