# Replicating a useful Stata Workflow with DataFramesMeta
**URL:** https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671
**Category:** Data
**Tags:** dataframes
**Created:** [January 29, 2018, 2:43pm UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671 "2018-01-29T14:43:58Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [January 29, 2018, 2:43pm UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/1 "2018-01-29T14:43:58Z")
One thing that is necessary when working with survey data is being able create new variables by standardizing capping existing variables variables. One workflow might look like this:
```julia
sysuse auto
local variables_to_cap price mpg headroom trunk weight
foreach variable in `variables_to_cap' {
gen `variable'_std = `variable' // generate a new variable but with a suffix added
sum `variable'_std, detail
replace `variable'_std = r(p99) if `variable'_std > r(p99) & !missing(`variable'_std)
}
```
Above, the suffixe `_std` indicates to the user that we are working with the standardized variable. The benefit of the above code is that you can add the suffix extremely easily, and use the exact same code to generate many standardized variables at once (Ideally, this would be done using a function.)
Using `DataFrames` we can’t do this because the way to refer to an existing variable is with a symbol, while the you create a variable
```julia
@transform(df, newCol = :oldCol) # Command(?) = Symbol works
@transform(df, symbol("newCol") = :oldCol) # error
@transform(df, :newCol) = :oldCol) # error
```
Is there a way to do this using DataFrames? Are there any proposals in the works to add this kind of functionality?
There may have been a thread about this recently but I couldn’t find it.
---
### Author:  [@nalimilan](https://discourse.julialang.org/u/nalimilan)
#### Post date: [January 29, 2018, 3:02pm UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/2 "2018-01-29T15:02:34Z")
Something like this works, but you lose the convenience of `@transform`:
```julia
transform(df; Symbol("newcol") => cos(df[:oldcol]))
```
You can also use `df[Symbol("newcol")] = df[:oldcol]` instead of `transform` (which is in DataFramesMeta, not in DataFrames BTW).
I’m not sure whether this has been discussed in DataFramesMeta yet, but you could file an issue about it, as it should be possible to support this in `@transform` in some way (maybe it already exists).
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [January 29, 2018, 3:44pm UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/3 "2018-01-29T15:44:55Z")
Thanks for the reply.
I think I will open an issue there, `Symbol("newcol")` is a bit ugly, but that’s fine if we have a leaner way of doing things in `DataFramesMeta`. Hopefully there is a clever workaround that makes this easy.
X-posted at DataFramesMeta [here](https://github.com/JuliaStats/DataFramesMeta.jl/issues/85)
---
### Author:  [@bramtayl](https://discourse.julialang.org/u/bramtayl)
#### Post date: [January 30, 2018, 4:46am UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/4 "2018-01-30T04:46:37Z")
Dynamically generated names are really hard to deal with. They will break type stability in something like Query and kinda make a mess of macro programming. Take a look at [Programming with dplyr](https://cran.r-project.org/web/packages/dplyr/vignettes/programming.html) for how dplyr attempts to solve this problem. dplyr also has mutate\_all or mutate\_each.
If you’re not pressed for performance, I think a better way to do this would be to gather the variables, create a new std column, gather the regular and the std column, then spread.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [January 30, 2018, 5:03am UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/5 "2018-01-30T05:03:49Z")
The absolute confusion of `quo` and `!!` in dplyr is something I would like to avoid. Stata gets around this by basically not having any other objects besides variables for the one dataset you are allowed to work with at a time, which is obviously limiting in other ways.
I think the main issue is the inconsistency between
```julia
@transform(df, newCol = f(:oldCol))
```
and
```julia
df[:newCol] = f(df[:oldCol]).
```
With DataFrames, you could have a vector of symbols `[:newVar1, :newVar2, ...]` and use that to create a bunch of new variables. There is no equivalent way to rename variables with DataFramesMeta. So that might be a task that takes precedence over any `suffix` commands.
---
### Author:  [@mwsohn](https://discourse.julialang.org/u/mwsohn)
#### Post date: [January 30, 2018, 6:44am UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/6 "2018-01-30T06:44:28Z")
How about this? This parallels the stata code closely.
```julia
variables_to_cap = [:price, :mpg, :headroom, :trunk, :weight]
for variable in variables_to_cap
newvar = Symbol(string(variable,"_std"))
q = quantile(df[variable],.99)
df[newvar] = [!isna(x) && x > q ? q : x for x in df[variable]]
end
```
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [January 30, 2018, 2:07pm UTC](https://discourse.julialang.org/t/replicating-a-useful-stata-workflow-with-dataframesmeta/8671/7 "2018-01-30T14:07:20Z")
I like this and it works, but it would be cool to do it all inside the DataFramesMeta ecosystem. I should have clarified in my title and original post.