Create temporary local namespace?

General Usage
1

In R you can use the following syntax to access variables within a data frame.

df = data.frame(x = 1:10, y = 1)
z = with(df, x + y) # => [2, 3, ..., 11]

The with function is very powerful in that it also allows you to use it like lisp’s let macro:

z = with(list(x = 1:10, y = 1), x + y) # => [2, 3, ..., 11]

with operates within the context of an environment in R, which is like a namespace.

Is there anything similar in Julia that allows you to work with DataFrame columns like this, or bind variables to local symbols?

2

You can just use dot syntax to access the columns as vectors.

julia> using DataFrames

julia> df = DataFrame(x=1:10, y=1)
10×2 DataFrame
 Row │ x      y
     │ Int64  Int64
─────┼──────────────
   1 │     1      1
   2 │     2      1
   3 │     3      1
   4 │     4      1
   5 │     5      1
   6 │     6      1
   7 │     7      1
   8 │     8      1
   9 │     9      1
  10 │    10      1

julia> z = df.x .+ df.y
10-element Vector{Int64}:
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11

or

julia> df.z = df.x .+ df.y
10-element Vector{Int64}:
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11

julia> df
10×3 DataFrame
 Row │ x      y      z
     │ Int64  Int64  Int64
─────┼─────────────────────
   1 │     1      1      2
   2 │     2      1      3
   3 │     3      1      4
   4 │     4      1      5
   5 │     5      1      6
   6 │     6      1      7
   7 │     7      1      8
   8 │     8      1      9
   9 │     9      1     10
  10 │    10      1     11

or

julia> transform!(df, [:x,:y] => ByRow(+) => :z)
10×3 DataFrame
 Row │ x      y      z
     │ Int64  Int64  Int64
─────┼─────────────────────
   1 │     1      1      2
   2 │     2      1      3
   3 │     3      1      4
   4 │     4      1      5
   5 │     5      1      6
   6 │     6      1      7
   7 │     7      1      8
   8 │     8      1      9
   9 │     9      1     10
  10 │    10      1     11
1 Like
3

What about the @with macro from DataFramesMeta?

3 Likes
4

Are you aware of let in Julia? Documentation, relevant manual section.

1 Like
5

Taking your post’s title at face value, let blocks binds local variables to global instances in its header and makes a (temporary) local scope to run the block in. But that’s not really what R is doing there. R is doing non-standard evaluation, which works more like implicit macros on argument expressions; unlike Julia, R arguments are lazily evaluated and make its way into the function as an expression you can opt to keep as one long enough for metaprogramming. Restricting names to a data.frame’s namespace is just one use case; it could also be used to provide strings for plot labels instead of evaluating variables.

Note that DataFramesMeta.@with requires you to write column symbols :x differently from existing variables x. That’s purposely engineered for the distinction, it does not imply that symbols are provided to macros with that syntax; in fact a simple @macroexpand x only sees :x in its body. The choice to incorporate the :x syntax also seems intended to clarify that you’re accessing a column name; a lot of confusion comes from R’s standard vs nonstandard evaluation looking the same at the call site, even being turned on and off by an optional argument.

3 Likes
6

DataFramesMeta.jl works very well for this

julia> using DataFramesMeta, Statistics

julia> df = DataFrame(x = rand(10), y = rand(10) .* 100);

julia> @with df begin
           mean(:x .+ :y)
       end
45.7031031775563

Doesn’t work on arbitrary lists, though (equivalent in Julia would be a Dict, more or less)

4 Likes
7

You can use property destructuring for this:

julia> list = (x=1:10, y = 1);

julia> z = let (;x, y) = list
           x .+ y
       end
2:11
4 Likes
8

You can also use @with from @Mason’s StaticModules.jl package:

It works on any object that supports both propertynames and getproperty, which includes structs, named tuples, and dataframes.

Although since he didn’t mention it himself, perhaps he prefers not to promote that package. :slight_smile:

4 Likes
10

Someone suggested your StaticModules package - this package is my favorite solution for the DataFrame problem Create temporary local namespace? - #8 by CameronBieganek. The destructuring bind you illustrate in your solution is also useful but then becomes cumbersome when you have a large number of columns in the DataFrame.

For the specific example I described, this is a very nice solution:

using StaticModules
z = @with df (x + y)

11

This seems the most general solution. It doesn’t differentiate between column names and global variables as does DataFramesMeta but I assume decides symbol values based on scoping rules - this is also how R works as well.

12

Thanks for the transform! example - this is separately useful (similar to R’s transform/mutate/within functions). The dot syntax is of course the obvious solution but my objective was to avoid writing the DataFrame name for each variable - largely for readability.