# Question about my data

**URL:** https://discourse.julialang.org/t/question-about-my-data/44237
**Category:** General Usage
**Tags:** question
**Created:** [August 4, 2020, 8:06am UTC](https://discourse.julialang.org/t/question-about-my-data/44237 "2020-08-04T08:06:50Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 8:06am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/1 "2020-08-04T08:06:50Z")

</div>

```julia
using CSV,Clustering,Statistics,DataFrames,TimeSeries,MarketData

data=CSV.read("E:\\测试数据\\huizong15min.csv")
heping_1MX_V_35kV=convert(Array,data[:,3])
xiangshan_1MX_V_35kV=convert(Array,data[:,4])
dafeng_1MX_V_110kV=convert(Array,data[:,5])
heping_1MX_V_110kV=convert(Array,data[:,6])
dafeng_dahe_P_220kV=convert(Array,data[:,7])
dafeng_1MX_V_220kV=convert(Array,data[:,8])
heping_1MX_V_220kV=convert(Array,data[:,9])
guilin_MX_V_500kV=convert(Array,data[:,10])
Thermal_Q=convert(Array,data[:,11])
Thermal_P=convert(Array,data[:,12])
Wind_Q=convert(Array,data[:,13])
Wind_P=convert(Array,data[:,14])
Load_Q=convert(Array,data[:,15])
Load_P=convert(Array,data[:,16])
guilin_guidu_P_220kV=convert(Array,data[:,17])
heping_MU_cos=convert(Array,data[:,18])
Water_S=convert(Array,data[:,19])

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/0/101688f942bb5aded0069f68c433efd4c12ce350.png)  
Why does my.CSV read out as “string”?thanks!

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 8:38am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/2 "2020-08-04T08:38:32Z")

</div>

What is the type of `data[:, 16]`? Also, why are you doing these conversions to `Array`? A column of a DataFrame is already a vector, so there’s no need to convert?

It also seems slightly unusual to decompose a DataFrame into a bunch of individual vectors like this, you might want to think about just renaming the columns of the DataFrame so that you can then access them by their names (e.g. `data.Thermal_Q`)

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 8:42am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/3 "2020-08-04T08:42:28Z")

</div>

This is data in hours, and I’m going to convert each of its columns into data in days (24 hours), which I say is the only way to facilitate data conversion

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 8:45am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/4 "2020-08-04T08:45:24Z")

</div>

And how do you convert a vector of type String to Float64

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 8:46am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/5 "2020-08-04T08:46:07Z")

</div>

But presumably this could be done in DataFrames as well, e.g.

```julia
combine(groupby(data, :Day), :Thermal_P => sum => :Thermal_P)

```

(assuming that your hour-to-day conversion is just summing over all hours, and that you have a column `Day` that gives you the day for each hour)

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 8:47am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/6 "2020-08-04T08:47:42Z")

</div>

`parse.(Float64, data.Load_P)`

But I’d have a look at your csv file - it’s likely that Excel has screwed up the formatting which is preventing CSV.jl from correctly reading the numbers. Open it up in Excel again and set the formatting to “General”, then save and reload with CSV, it should correctly parse the numbers (assuming that you don’t have non-numerical data in the same column)

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 8:50am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/7 "2020-08-04T08:50:17Z")

</div>

I need all the data, just rearrange them（a-\>{M_1} into a-\>{N_24}）

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 8:57am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/8 "2020-08-04T08:57:32Z")

</div>

It’s still not in the right format 😣

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 8:58am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/9 "2020-08-04T08:58:29Z")

</div>

That’s unfortunate, but also pretty much impossible to debug remotely unless you can share the csv file or some dummy data that reproduces your issue.

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:01am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/10 "2020-08-04T09:01:26Z")

</div>

How can I send my data to you?

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 9:03am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/11 "2020-08-04T09:03:42Z")

</div>

Depends on how large it is and whether you can make it public or not - you could just upload it on GitHub or some other filesharing service, or if you can’t make it public and it’s not too large I can DM you an email address to send it to

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:04am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/12 "2020-08-04T09:04:37Z")

</div>

Ok, I will send it to your email，thankyou

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:05am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/13 "2020-08-04T09:05:26Z")

</div>

Would you mind giving me your E-mail address？

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 9:05am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/14 "2020-08-04T09:05:42Z")

</div>

Sent you a DM

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:11am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/15 "2020-08-04T09:11:31Z")

</div>

I have sent it to you, 😁

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 9:21am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/16 "2020-08-04T09:21:14Z")

</div>

Thanks - the issue you have is that there is one row in your data where missing values are encoded not with an empty cell, but with a single whitespace character. The simplest thing to do is to just use Ctrl+H (find/replace) in Excel to replace `" "` in your file with `""` (i.e. put a single empty character in the top box and nothing in the bottom box). This should make 12 replacements in your file, and if you then do

```julia
df = DataFrame(CSV.File("huizong15min.csv"))

```

you should get all numerical columns (although note that you’ll have `Union{Float64, Missing}` data type, as there are missing values now, you can use `dropmissing!(df)` to get rid of those)

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:54am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/17 "2020-08-04T09:54:03Z")

</div>

I did as you asked, but nothing seems to have changed,And it’s not found in Excel " "

---

<div class="post-metadata">

### Author: ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [August 4, 2020, 9:56am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/18 "2020-08-04T09:56:46Z")

</div>

The problem is in row 17,702 (timestamp 43648.38542), where almost all of your data is missing. Just manually delete the content of the empty cells (or the whole row if you would drop it afterwards anyway)

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:58am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/19 "2020-08-04T09:58:22Z")

</div>

Sorry, that was my formatting error, I have corrected it, that column should be the date

---

<div class="post-metadata">

### Author: ![abcde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abcde/32/16180_2.png) [@abcde](https://discourse.julialang.org/u/abcde)
#### Post date: [August 4, 2020, 9:59am UTC](https://discourse.julialang.org/t/question-about-my-data/44237/20 "2020-08-04T09:59:12Z")

</div>

![image](https://global.discourse-cdn.com/julialang/original/3X/2/e/2ea12ce47b44897ce0d91be1a1b31880a0952292.png)

[Next page](https://discourse.julialang.org/t/question-about-my-data/44237.md?page=2)
