### Author:  [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)
#### Post date: [October 24, 2023, 7:17am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/1 "2023-10-24T07:17:06Z")
Is there an easy way to transform several columns of a table using the same MLJ transformer? So if I have a transformer like `UnivariateBoxCoxTransformer`, I’d like to apply it only to columns 2, 3, and 4.
Alternatively, some way to avoid applying a transformer (like `Standardizer`) to some columns.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 24, 2023, 8:09am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/2 "2023-10-24T08:09:05Z")
If you can’t find a solution with MLJ.jl take a look at TableTransforms.jl.
---
### Author:  [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)
#### Post date: [October 24, 2023, 9:13pm UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/3 "2023-10-24T21:13:06Z")
Thanks, but unfortunately, I’m looking for an answer to make a PR to MLJ.jl, and I’d rather not add a dependency 😅
---
### Author:  [@ablaom](https://discourse.julialang.org/u/ablaom)
#### Post date: [October 24, 2023, 10:26pm UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/4 "2023-10-24T22:26:36Z")
I’ve long thought we should have this, but we don’t. An issue was opened some time ago:
>
>
> It has been proposed on Slack that it be possible to have a single table transfo…rmer that transforms individual columns according to \*\*user-specified\*\* univariate transformations. This sounds like a good idea, which would also force some uniformity that's a little bit lacking in the current collection of table transformers.
>
> 1. In the most general case I can imagine implementing, the univariate transformer that applies to a particular column is defined by a function that operates on both the \`name\` and \`scitype\` of the the column (as encoded in the table \`schema\`). This has the disadvantage that the user must specify a function with two arguments - or interact through some other complicated interface.
>
> 2. The alternative would be a compositional approach. Each tabular transformer only carries out a \*single\* univariate transformer, applying to all specified \`names\` and \`scitypes\` (or "not"-names and "not"-scitypes, through \`ignore\` Boolean parameter), which would cover all conceivable use-cases. (columns not referred to are left alone). However, as we are currently locked into Tables.jl (which are non-mutable in general) we get a lot more copying of data.
>
> Thoughts anyone?
---
### Author:  [@math4mad](https://discourse.julialang.org/u/math4mad)
#### Post date: [October 25, 2023, 12:40am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/5 "2023-10-25T00:40:50Z")
Please! It is very cumbersome, like this
```julia
# 1. loading package
using DataFrames,TableTransforms
using Random
Random.seed!(34343)
# 2. loading data -> dataframe
df=load_csv("BostonHousing")
# 3. tranformation
table=(eachcol(df))|>Functional(
1=>log,
2=>x->x/10,
3=>log,
4=>x->x,
5=>log,
6=>log,
7=>x->(x^2.5)/10000,
8=>log,
9=>log,
10=>log,
11=>x->exp(0.4*x)/1000,
12=>x->x/100,
13=>x->sqrt(x),
14=>log
)
first(table,10)
```
[1,3,5,6,8,9,10,14] better be one line
---
### Author:  [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)
#### Post date: [October 25, 2023, 3:15am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/6 "2023-10-25T03:15:20Z")
Damn. So TIL about `Functional`, which is a huge quality-of-life improvement already. Actually, as I’m reading more about TableTransforms.jl, I’m thinking this is a great consolidation opportunity; most of the built-in transformations in MLJModels.jl might fit better in TableTransforms.
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 25, 2023, 10:29am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/7 "2023-10-25T10:29:59Z")
> [@math4mad](#):
>
> [1,3,5,6,8,9,10,14] better be one line
TableTransforms.jl has very flexible column selection features:
```julia
Functional([1,3,5,6,8,9,10,14] => log)
```
You can use lists of symbols, strings, integers, regex, …
---
### Author:  [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)
#### Post date: [October 25, 2023, 4:57pm UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/8 "2023-10-25T16:57:58Z")
> [@juliohm](#):
>
> ```julia
> Functional([1,3,5,6,8,9,10,14] => log)
>
> ```
Ooh, neat! Perhaps this could be more clearly documented?
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 25, 2023, 5:04pm UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/9 "2023-10-25T17:04:12Z")
All transforms should have a clear docstring explaining these options.
---
### Author:  [@math4mad](https://discourse.julialang.org/u/math4mad)
#### Post date: [October 27, 2023, 6:32am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/10 "2023-10-27T06:32:19Z")
this method doesn’t work right now
---
### Author:  [@juliohm](https://discourse.julialang.org/u/juliohm)
#### Post date: [October 27, 2023, 11:22am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/11 "2023-10-27T11:22:03Z")
You need to be more explicit about which method doesn’t work. Can you please share a MWE?
---
### Author:  [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)
#### Post date: [October 29, 2023, 5:04pm UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/12 "2023-10-29T17:04:29Z")
Is there a way to use TableTransforms.jl together with MLJ transforms?
---
### Author:  [@ablaom](https://discourse.julialang.org/u/ablaom)
#### Post date: [October 30, 2023, 2:32am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/13 "2023-10-30T02:32:20Z")
Not currently. The main issues are:
1. ( **abstract type roadblock** ) MLJModelInterface requires new algorithms to subtype an abstract type owned by MLJModelInterface (`Unsupervised` or `Static`) but TableTransforms.jl, as I understand it, is trying for a pure functional interface, and without depending on externally owned types.
2. ( **limitation on functionality** ) The MLJTuning.jl API for tuning models is based on mutation of the hyperparameter struct, and so not suited to TableTransforms.jl transformer structs, which are immutable. This currently rules out optimization of transformer hyperparameters in MLJ pipelines.
One day MLJ may rid itself of its abstract model type hierarchy (for efforts in this direction, see [this announcement](https://discourse.julialang.org/t/ann-learnapi-jl-proposal-for-a-basement-level-machine-learning-api/93048/1)). However, it is substantially embedded in the ecosystem and unlikely to disappear in the near future.
A simple, but unattractive, solution to 1. would be for MLJModels.jl or TableTransforms.jl to provide a wrapper. The only way I can think of to avoid the wrapper in the status quo would require metaprogramming hacks that would likely be brittle.
---
### Author:  [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)
#### Post date: [November 2, 2023, 1:02am UTC](https://discourse.julialang.org/t/transform-several-columns-of-an-mlj-model-using-one-transformer/105338/14 "2023-11-02T01:02:39Z")
> [@ablaom](#):
>
> A simple, but unattractive, solution to 1. would be for MLJModels.jl or TableTransforms.jl to provide a wrapper. The only way I can think of to avoid the wrapper in the status quo would require metaprogramming hacks that would likely be brittle.
Sounds good to me!