# \[ANN\] StaticModules.jl
**URL:** https://discourse.julialang.org/t/ann-staticmodules-jl/48010
**Category:** Package Announcements
**Created:** [October 9, 2020, 1:15am UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010 "2020-10-09T01:15:30Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2020, 1:15am UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/1 "2020-10-09T01:15:30Z")
Hi all, I recently made a little package [StaticModules.jl](https://github.com/MasonProtter/StaticModules.jl) (awaiting registration in 3 days).
The idea is that a `StaticModule` is kinda like a `module`, but can be created in the local scope, is immutable and doesn’t support things like `using`. Basically a fancy `NamedTuple` that you can ‘run code inside’.
For example,
```julia
julia> using StaticModules
julia> @staticmodule Foo begin
x = 1; y = x^2 + 2x - 1
f(z) = (x^2 + 2y)/z
end
StaticModule Foo containing
f = f
y = 2
x = 1
julia> f(x/y)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] top-level scope at REPL[7]:1
julia> @with Foo begin
f(x/y)
end
10.0
```
Furthermore, the `@with` macro will work with _any_ type that’ll give you values from `getproperty`, e.g.
```julia
julia> nt = (;a=1, b="hi")
(a = 1, b = "hi")
julia> @with nt begin
a + length(b)
end
3
```
Importantly, you should be able to use `@staticmodule` and `@with` without suffering any runtime performance penalty, so this can be used even in things like tight-loops if you want to namespace some code.
I kinda see this as offering similar (though different) features from packages like [Parameters.jl](https://github.com/mauro3/Parameters.jl). One advantage this has is that `StaticModules.@with` will work on arbitrary structs and `NamedTuples`, whereas `Parameters.@unpack` requires that you either register a struct with Parameters.jl, or tell it what symbols to unpack from a struct or named tuple. @mauro3, if you’re interested, I think `StaticModules.@with` can be lifted for Parameters.jl pretty easily, though it’d add new dependancies.
All comments, questions, suggestions or bikeshedding welcome!
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2020, 4:34pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/2 "2020-10-09T16:34:40Z")
One thing under consideration right now is [https://github.com/MasonProtter/StaticModules.jl/issues/3](https://github.com/MasonProtter/StaticModules.jl/issues/3). I’d be interested if any other potential users have thoughts or suggestions on this.
---
### Author:  [@ianfiske](https://discourse.julialang.org/u/ianfiske)
#### Post date: [October 9, 2020, 5:46pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/3 "2020-10-09T17:46:28Z")
This looks really cool! Sort-of reminds me of R’s `with` function and I was wanting something like this very recently.
What do you think of supporting `@with` on multiple static modules and/or objects-with-getproperty. Like this perhaps?
```julia
df = DataFrame(x=rand(10))
struct Bar
a
b
end
bar = Bar(1,2)
@with df, bar begin
some_code(x, b)
end
```
In case of name-collisions, some merging and precedence would need to be handled.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2020, 6:15pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/4 "2020-10-09T18:15:41Z")
Yeah, good suggestion I was actually thinking about that one this morning.
Currently, the way `@with` works is that
```julia
@with foo begin
x = 1 + y
z = f(x)
end
```
it’ll detect that that `:y` and `:f` are symbols from outside the scope of the block, so it’ll turn this into
```julia
let y = (:y in propertynames(foo) ? foo.y : y), f = (:f in propertynames(foo) ? foo.f : f)
x = 1 + y
z = f(x)
end
```
Hence, I think the easiest thing to do would be to make it turn
```julia
@with (foo, bar) begin
x = 1 + y
z = f(x)
end
```
into
```julia
let y = (:y in propertynames(foo) ? foo.y : :y in propertynames(bar) ? bar.y : y), f = (:f in propertynames(foo) ? foo.f : :f in propertynames(bar) ? bar.f : f)
x = 1 + y
z = f(x)
end
```
If everything is inferrable, it should be possible to eliminate these `if/else` blocks at compile time, but the more names in the `@with` the higher the chances are that the compiler gives up.
The way this would handle name collision is that whichever thing comes first has priority (e.g. `foo` has priority over `bar`)
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [October 9, 2020, 8:33pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/5 "2020-10-09T20:33:56Z")
> [@ianfiske](#):
>
> What do you think of supporting `@with` on multiple static modules and/or objects-with-getproperty. Like this perhaps?
> [@Mason](#):
>
> If everything is inferrable, it should be possible to eliminate these `if/else` blocks at compile time, but the more names in the `@with` the higher the chances are that the compiler gives up.
Just a note with regards to DataFrames, unfortunately this isn’t possible to make performant with DataFrames. See [this post](https://discourse.julialang.org/t/understanding-the-performance-costs-of-generating-functions-on-the-fly/47990/10) outlining a similar feature yesterday.
Since the property names of a dataframe aren’t inferrable, any expression whose final form depends on the types in an `if/else` way won’t have all the optimizations available that normal functions have. If you don’t want special designations for columns, i.e. `df.x` referenced by the `Symbol` `:x` then you would need to treat every “variable” in the expression as a column. This can get complicated very quickly.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2020, 8:40pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/6 "2020-10-09T20:40:17Z")
Yes, how could it be otherwise? This is a general problem with dataframes, they might as well be a `Dict{Symbol, Any}`. Any sort of zero-cost static abstraction like this is going to pay a performance price for untyped stuff like DataFrames or Dicts.
Of course, if you’re working on a dataframe, presumably things like `getproperty` shouldn’t be your bottleneck anyways and if something like this is causing you a significant bottleneck, that’s a good indication that a dataframe is the wrong tool for the job you’re doing.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [October 9, 2020, 8:43pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/7 "2020-10-09T20:43:52Z")
Just to be clear, in current DataFramesMeta (on `master` and the release branch), we always know which parts of the expression represent columns in the data frame. So the code-generation is as fast as just taking out the columns individually and using a function that is defined as compile time. So the benefit of DataFramesMeta is that you _can_ use a data frame for this and we get to pretent, as much as possible, that the propertynames and types are known.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2020, 8:48pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/8 "2020-10-09T20:48:51Z")
Right, but in `DataFramesMeta.@with` the user is expressedly marking for you which symbols are keys of the DataFrame. And there’s still a runtime penalty because you need to actually run `getproperty(df, :x)`.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [October 9, 2020, 8:52pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/9 "2020-10-09T20:52:58Z")
Yes, exactly. That should be very cheap, of course, especially relative to the computation. Additionally in `@byrow` you only pay that penalty once, even though we loop through all rows.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 9, 2020, 9:41pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/10 "2020-10-09T21:41:59Z")
Okay, I implemented this, now on master it works like this:
```julia
julia> nt = (;a = 1, b=2)
(a = 1, b = 2)
julia> struct Bar; b; c end
julia> @with nt, Bar("hi", "bye") begin
a, b, c
end
(1, 2, "bye")
```
---
### Author:  [@zennatavares](https://discourse.julialang.org/u/zennatavares)
#### Post date: [October 9, 2020, 10:50pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/11 "2020-10-09T22:50:31Z")
Can static modules be created dynamically, and will they be garbage collected? I ask because I’ve been doing some program synthesis, which involves repeatedly generating modules. Since normal modules are not garbage collected I have to do a lot of hackery to handle the memory leaks.
---
### Author:  [@zennatavares](https://discourse.julialang.org/u/zennatavares)
#### Post date: [October 9, 2020, 10:54pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/12 "2020-10-09T22:54:07Z")
Seems like maybe StaticModules won’t help my use case? I need to eval generated code and that needs to occur within a real module.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 10, 2020, 6:37am UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/13 "2020-10-10T06:37:28Z")
> [@zennatavares](#):
>
> Can static modules be created dynamically, and will they be garbage collected?
Yes and yes. In the case of garbage collection, there’s not actually anything to garbage collect unless you allocate objects inside the static module that needs to be garbage collected. Those objects will be garbage collected as normal once they’re not needed by anything.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 10, 2020, 6:38am UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/14 "2020-10-10T06:38:37Z")
That depends. Do you really need `eval`? Are you sure you can’t use a macro?
---
### Author:  [@ianfiske](https://discourse.julialang.org/u/ianfiske)
#### Post date: [October 10, 2020, 12:37pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/15 "2020-10-10T12:37:30Z")
Wow, this is fantastic! Thank you for the quick implementation.
---
### Author:  [@rfourquet](https://discourse.julialang.org/u/rfourquet)
#### Post date: [October 10, 2020, 12:59pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/16 "2020-10-10T12:59:08Z")
Looks like a nice feature! Since you are open to bikeshedding…
- actually, what is “static” referring to? As the first line of the README starts with
- just a thought, `@within` could sound nice (as an alternative to `@with`), e.g.
```julia
julia> @within Foo begin
f(1) == 3x
end
```
(but maybe not so nice when the argument is not a staticmodule, but a regular object like a named-tuple).
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 10, 2020, 3:58pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/17 "2020-10-10T15:58:25Z")
> [@rfourquet](#):
>
> - actually, what is “static” referring to? As the first line of the README starts with
>
> > a StaticModule is basically a little namespace
> > Why not call them “namespaces”, e.g. with a `@namespace` macro?
I actually originally was going to call it Namespaces.jl, but changed over to StaticModules.jl because I think it’s a useful analogy to StaticArrays.jl. Just like how a `StaticArray` is backed a `Tuple`, a `StaticModule` is backed by a `NamedTuple`. `StaticModule`s are hence immutable and the names of the values defined in them are compile time constants as well as the types of the variables those names refer to. e.g.
```julia
julia> using StaticModules
julia> @staticmodule Foo begin
x = 1
f(y) = x^2 + 2y
end
StaticModule Foo containing
f = f
x = 1
julia> typeof(Foo)
StaticModule{:Foo,(:f, :x),Tuple{var"#f#1"{Int64},Int64}}
```
Just like how StaticArrays will cause long compile times and even runtime problems if they get too long, a StaticModule will cause the same problems if you store too many variables in it, so I figured the name `StaticModule` was more appropriate than `Namespace`.
Perhaps I should have been more clear about this in the README.
> [@rfourquet](#):
>
> - just a thought, `@within` could sound nice (as an alternative to `@with` ), e.g.
>
> ```julia
> julia> @within Foo begin
> f(1) == 3x
> end
>
> ```
>
> (but maybe not so nice when the argument is not a staticmodule, but a regular object like a named-tuple).
Yeah maybe. I’ll think about this one. Maybe I should spell it like
```julia
@within Foo do
f(1) == 3x
end
```
rather than with `begin`?
---
### Author:  [@zennatavares](https://discourse.julialang.org/u/zennatavares)
#### Post date: [October 10, 2020, 8:32pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/18 "2020-10-10T20:32:39Z")
I could be wrong, but I don’t see how I could not need to eval? The synthesis algorithms (dynamically) generate code, as in `Expr` values, it `eval`s them, then it execute them.
---
### Author:  [@Mason](https://discourse.julialang.org/u/Mason)
#### Post date: [October 10, 2020, 8:49pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/19 "2020-10-10T20:49:46Z")
That depends greatly on your actual use case and needs. Just saying ‘program synthesis’ is sufficiently vague that it’s hard to make any substantive comments.
If you want to open a separate Discourse or Zulip thread on the issues you’re experiencing with some MWEs, I’d be happy to offer advice on this. I’ve thought a fair amount about dynamic code generation (though others have thought far more than me)
There are a lot of devious techniques out there to avoid `eval`.
---
### Author:  [@zennatavares](https://discourse.julialang.org/u/zennatavares)
#### Post date: [October 10, 2020, 8:52pm UTC](https://discourse.julialang.org/t/ann-staticmodules-jl/48010/20 "2020-10-10T20:52:49Z")
Sure
[Next page](https://discourse.julialang.org/t/ann-staticmodules-jl/48010.md?page=2)