# Training dataset and reporting MSE

**URL:** <https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391>\
**Category:** Machine Learning\
**Created:** [February 10, 2020, 12:18am UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391 "2020-02-10T00:18:34Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Praise](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/praise/32/13948_2.png) [@Praise](https://discourse.julialang.org/u/Praise)\
**Post date:** [February 10, 2020, 12:18am UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/1 "2020-02-10T00:18:34Z")

</div>

Hello, I am trying to train dataset using linear regression in julia and then I have to calculate the MSE of the given test dataset. I was wondering what are the best packages to use, and an example of how to do this with a .csv file would really be helpful.

Also, the csv file I have doesn’t have column names, instead has numbers there and there is Float64 written below it. It is like the table I have shown below, I just copy pasted some of the .csv file data just to give an idea, but yes this is the kind of .csv file I have to use to train and calculate MSE.

1.321249999999999855e-02 2.695833333333333501e-03 -2.254166666666667339e-03 3.345833333333333472e-03  
Float64 Float64 Float64 Float64  
1 -0.0095125 0.00390417 -0.0012125 0.00287083  
2 0.0005875 0.0030625 -0.0015625 0.00432083  
3 0.0139125 0.0010375 0.000845833 -0.00387917  
4 -0.0028625 0.0017125 0.00257917 0.00325417  
5 -0.0145625 -0.0005125 0.0013125 0.00390417  
6 -0.0052875 0.0023875 0.000320833 -0.000854167  
7 0.0083125 0.0030125 0.00204583 -0.0004625  
8 0.0012125 -0.00314583 -0.00370417 -0.0012125  
9 0.0009875 0.00402917 -7.91667e-5 -0.00345417  
10 0.0125125

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 10, 2020, 11:08am UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/2 "2020-02-10T11:08:18Z")

</div>

This is a pretty broad question, you might want to try some things yourself to narrow it down to any issues you might eventually face.

Starting points:

```julia
using CSV # library to read CSV files

df = CSV.read("myfile.csv")

using GLM # package to fit linear models

lm(@formula(y ~ x1 + x2 + x3), df) # fit a linear model, assuming your data has columns called y, x1, x2, and x3

```

This is the simplest way - as you talk about “training” a model you might be looking to fit a model through some sort of optimization rather than just fitting OLS, in which case you should look at ML libaries. I’d probably start with [`MLJ.jl`](https://alan-turing-institute.github.io/MLJ.jl/stable/) which offers a number of models to fit with a unified interface.

---

<div class="post-metadata">

**Author:** ![Praise](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/praise/32/13948_2.png) [@Praise](https://discourse.julialang.org/u/Praise)\
**Post date:** [February 10, 2020, 2:35pm UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/3 "2020-02-10T14:35:13Z")

</div>

Yes, that certainly helps a lot. I am still new to Julia and this at least tells me I was on the right path. Thank you!

You said “assuming your data has columns called y, x1, x2, and x3”. My data in the .csv file has weird column names. I have copy pasted a bit of the sample code below where the numbers above Float64 are the column names (Ex: here it is 1.321249999999999855e-02) and Float64 is just the type and so on.

1.321249999999999855e-02 2.695833333333333501e-03 -2.254166666666667339e-03  
Float64 Float64 Float64  
1 -0.0095125 0.00390417 -0.0012125  
2 0.0005875 0.0030625 -0.0015625

I wanted to know if we can change the column names in Julia, or do we have to do this by opening it in another application?

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 10, 2020, 2:39pm UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/4 "2020-02-10T14:39:52Z")

</div>

I guess your csv doesn’t have headers? You can do a number of things:

1. Manually add a header line to the csv (just add a row in Excel or something and type names for columns into it)

2. Pass the header directly into `CSV.read` like `CSV.read("myfile.csv", header = ["first_column_name", "second_column_name", "third_column_name"])`

3. If you’ve read the file into a DataFrame (let’s call it `df`) already you can call `rename!(df, [:first_column_name, :second_column_name, :third_column_name])` on it

---

<div class="post-metadata">

**Author:** ![Praise](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/praise/32/13948_2.png) [@Praise](https://discourse.julialang.org/u/Praise)\
**Post date:** [February 10, 2020, 2:47pm UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/5 "2020-02-10T14:47:26Z")

</div>

The second solution worked for me perfectly.

Thank you so much for the help, I appreciate it. Now the whole thing makes a lot more sense.

---

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [February 10, 2020, 2:48pm UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/6 "2020-02-10T14:48:08Z")

</div>

If you’re new to julia and want to use it for ML, consider following the julia academy courses here. [https://juliaacademy.com/](https://juliaacademy.com/) There’s one on machine learning and the first lectures are about datasets etc.

---

<div class="post-metadata">

**Author:** ![Praise](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/praise/32/13948_2.png) [@Praise](https://discourse.julialang.org/u/Praise)\
**Post date:** [February 10, 2020, 2:49pm UTC](https://discourse.julialang.org/t/training-dataset-and-reporting-mse/34391/7 "2020-02-10T14:49:16Z")

</div>

I will certainly check it out! Thanks for the input.