# Standardize dataset with StatsBase

**URL:** <https://discourse.julialang.org/t/standardize-dataset-with-statsbase/37034>\
**Category:** Machine Learning\
**Created:** [April 4, 2020, 7:51pm UTC](https://discourse.julialang.org/t/standardize-dataset-with-statsbase/37034 "2020-04-04T19:51:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![alequa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alequa/32/12338_2.png) [@alequa](https://discourse.julialang.org/u/alequa)\
**Post date:** [April 4, 2020, 7:51pm UTC](https://discourse.julialang.org/t/standardize-dataset-with-statsbase/37034/1 "2020-04-04T19:51:18Z")

</div>

Hi,  
I am trying to apply this [standardization scheme](https://stats.stackexchange.com/questions/202287/why-standardization-of-the-testing-set-has-to-be-performed-with-the-mean-and-sd) where you standardize the test set on the base of the train set.  
I am trying to do this with the ZScoreTransform of StatsBase, but I encounter this trouble.

```julia
#

using Distributions
using StatsBase
#(features, datapoints)
train = rand(Normal(1,10),(100,1000))
test = rand(Normal(1,5),(100,100))

## Verify the train dataset is normalized
println("before standardization")
mean(train,dims=2)
std(train,dims=2)
mean(test,dims=2)
std(test,dims=2)
## Train the ZScoreTransform
train_std = StatsBase.fit(ZScoreTransform, train , dims=2)
StatsBase.transform!(train_std,train)

## And each feature get standardized
mean(train,dims=2)
std(train,dims=2)

# Then I want to standardize the test set.
StatsBase.transform!(train_std,test)
# But I get Error Dimension mismatch!!!! Despite the mean and scale are with the correct size!!
size(train_std.mean)
size(train_std.scale)

```

Why? And how do I solve it?

---

<div class="post-metadata">

**Author:** ![ppalmes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ppalmes/32/6838_2.png) [@ppalmes](https://discourse.julialang.org/u/ppalmes)\
**Post date:** [April 4, 2020, 8:00pm UTC](https://discourse.julialang.org/t/standardize-dataset-with-statsbase/37034/3 "2020-04-04T20:00:00Z")

</div>

i run the code, there was no problem. maybe update to latest version?