# Converting CSV string values to floats (Python to Julia)
**URL:** https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330
**Category:** New to Julia
**Tags:** python, dataframes, csv
**Created:** [January 31, 2021, 3:27pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330 "2021-01-31T15:27:58Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [January 31, 2021, 3:27pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/1 "2021-01-31T15:27:58Z")
I am very new to Julia and I am struggling with the following. I have a python hdf5 file in Python full of data and I convert it to a Python bumpy DataFrame which I then save to a csv file. Then I want to use this file in Julia but it looks much different than in Python:
[Screen Shot 2021-01-31 at 11.25.54 PM|497x499](https://discourse.julialang.org/uploads/short-url/gD917uD1SJ84QBZGvU025q17w9J.jpeg)
1. Python’s “0” index is converted to a header when I read the csv file in Julia
2. In python I have a dataframe 100x1 but in Julia this gets converted to 99x2.
See attached images.
Furthermore, in Julia the data is converted into strings which means I cannot use them really. Can somebody help to figure this out?
I load the file in Julia using:
`data = CSV.read("data.csv", DataFrame);`
I also attach the csv here: [https://drive.google.com/file/d/10KbvkyYv1Bmg4Kez2oRyuXiCgunDDUtH/view?usp=sharing](https://drive.google.com/file/d/10KbvkyYv1Bmg4Kez2oRyuXiCgunDDUtH/view?usp=sharing)
---
### Author:  [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)
#### Post date: [January 31, 2021, 4:08pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/2 "2021-01-31T16:08:20Z")
You should be able to load the HDF5 file directly using HDF5.jl. If that’s not an option, it’d help if you could provide the raw CSV file (or at least the first few rows) and whatever commands you’re currently using to load it. See this post for pointers on how to make your question easier to answer:
> [@Please read: make it easier to help you](https://discourse.julialang.org/t/psa-make-it-easier-to-help-you/14757):
>
> Welcome to the Julia Discourse! We are enthusiastic about helping Julia programmers, both beginner and experienced. This public service announcement (PSA) outlines best practices when asking for help. Following these points makes it easier for us to help you and more likely you’ll get a prompt, useful answer. Keywords are highlighted to make it easier to refer to specific points. Choose a descriptive title that captures the key part of your question, eg “plots with multiple axes” instead of …
---
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [January 31, 2021, 4:15pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/3 "2021-01-31T16:15:43Z")
I just did as you said. I use the standard `data = CSV.read("data.csv", DataFrame);` command and I linked the csv file.
---
### Author:  [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)
#### Post date: [January 31, 2021, 4:38pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/4 "2021-01-31T16:38:23Z")
Your CSV file isn’t publicly visible with your current sharing settings. If you already have a GitHub account, the easiest thing is to upload it to [gist.github.com](http://gist.github.com).
---
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [January 31, 2021, 5:11pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/5 "2021-01-31T17:11:35Z")
Check again please.It is now.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [January 31, 2021, 5:23pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/6 "2021-01-31T17:23:39Z")
1. Use the keyword argument `header = false` in `CSV.read`
2. Figure out how to use `parse` the column to an object of type `Complex{Float64}`, i.e.
```julia
julia> y = "65.4 + 98.2im"
"65.4 + 98.2im"
julia> parse(Complex{Float64}, y)
65.4 + 98.2im
```
What you need to figure out how to do is how to “clean” the values of your data frame so that they work with `parse`.
This involves
1. `strip` to remove extra white space
2. using `replace` to remove the `(` and `)`.
Here is a full example
```julia
julia> using CSV, Chain, DataFrames
julia> df = CSV.read("data.csv", DataFrame; delim = ",", header = false);
julia> function clean_parse_complex(x)
c = @chain x begin
strip()
replace("(" => "")
replace("(" => "")
parse(Complex{Float64}, _)
end
end
clean_parse_complex (generic function with 1 method)
julia> df.c = clean_parse_complex.(df.Column1);
```
One thing to note, though, is it looks like all your values are real! They all have `0` imaginary component. Maybe you are encoding things as complex in python when that isn’t necessary?
---
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [January 31, 2021, 5:25pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/7 "2021-01-31T17:25:32Z")
This is part of my data. I have more with complex values. I will try and see if it works. Not sure how I could figure all this out without assistance here.
---
### Author:  [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [January 31, 2021, 5:27pm UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/8 "2021-01-31T17:27:22Z")
The process of “figuring it out” should be the same as in any language. If you want to parse a string, you have to clean it up a bit first.
From there it’s just a matter of using the `?` tool in the command line. `? parse`, `? replace` in order to nail the syntax
---
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [February 2, 2021, 6:57am UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/9 "2021-02-02T06:57:01Z")
Thanks a lot. I am trying to use the function that you defined above for a second dataset which actually involves imaginary parts. Nevertheless I get the error:
`ArgumentError: expected trailing "im", found only "m"`
`Stacktrace: [1] tryparse_internal(::Type{Complex{Float64}}, ::String, ::Int64, ::Int64, ::Bool) at ./parse.jl:316 [2] parse(::Type{Complex{Float64}}, ::String) at ./parse.jl:378 [3] clean_parse_complex(::String) at ./In[46]:11 [4] _broadcast_getindex_evalf at ./broadcast.jl:648 [inlined] [5] _broadcast_getindex at ./broadcast.jl:621 [inlined] [6] getindex at ./broadcast.jl:575 [inlined] [7] macro expansion at ./broadcast.jl:932 [inlined] [8] macro expansion at ./simdloop.jl:77 [inlined] [9] copyto! at ./broadcast.jl:931 [inlined] [10] copyto! at ./broadcast.jl:886 [inlined] [11] copy at ./broadcast.jl:862 [inlined] [12] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(clean_parse_complex),Tuple{Array{String,1}}}) at ./broadcast.jl:837 [13] top-level scope at In[46]:16 [14] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091`
I cannot understand why it works for the first dataset but not the second one. I attach here the second dataset in case you can provide some assistance.
> **[density\_1.csv](https://drive.google.com/file/d/1AeFLJkTR_jg1RsALTu4rSq6I8OZDWXNu/view?usp=sharing)**
>
> Google Drive file.
---
### Author:  [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)
#### Post date: [February 2, 2021, 7:15am UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/10 "2021-02-02T07:15:01Z")
For one thing, it has a header - you’ll need to change the `header` keyword argument for `CSV.read` from `false` to `true`.
```julia
julia> df = CSV.read("density_1.csv", DataFrame; delim=',');
julia> function clean_parse_complex(x)
c = @chain x begin
strip()
replace("(" => "")
replace(")" => "")
replace("j" => "im")
parse(Complex{Float64}, _)
end
end
julia> clean_parse_complex.(df."Element 01 of state |0>")
100-element Vector{ComplexF64}:
-0.02587890624999966 - 0.013916015624999997im
-0.005371093750000201 - 0.00024414062499999087im
-0.02124023437499952 + 0.021728515624999976im
-0.0031738281249997355 + 0.013427734375000016im
-0.02172851562499974 + 0.001708984374999981im
...
```
---
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [February 2, 2021, 7:32am UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/11 "2021-02-02T07:32:19Z")
Thanks. So, I figured it out my self. Initially I was saving my data as csv using the following:
`np.savetxt("density_3.csv", df, delimiter=",", fmt='%s')`
in the example I initiated this topic with. Then I realized that the rest of my dataset was saved as csv files using:
`df.to_csv('ata.csv', index=False)`
As mentioned above indeed, the latter way includes a header that I did not know how to get rid off initially. But it all works now. Quite frustrating for a beginner.
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [February 2, 2021, 9:25am UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/12 "2021-02-02T09:25:19Z")
Beginning to learn a programming language by parsing strings doesn’t sound like a great idea to me. (And if you do, I don’t see how Julia is any more difficult in that regard than other languages.)
As has already been pointed out above, it is probably a mistake to convert your data to from HDF to CSV and then parse it in Julia. Why not use [HDF5.jl](https://github.com/JuliaIO/HDF5.jl) like you’re using `h5py` in python directly? This will preserve all the proper data types and string parsing won’t be necessary at all.
Alternatively, you could try to convert a python data frame (pandas at least) directly to a Julia `DataFrame` via [PyCall.jl](https://github.com/JuliaPy/PyCall.jl) and/or [Pandas.jl](https://github.com/JuliaPy/Pandas.jl). This would again avoid any csv business.
---
### Author:  [@George\_Korpas](https://discourse.julialang.org/u/George_Korpas)
#### Post date: [February 2, 2021, 9:50am UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/13 "2021-02-02T09:50:17Z")
Thanks. I 've been using symbolic tools for years and programming for such also for years but now only I have to deal with proper data. So, sometimes, I am unaware even of the proper terminology, e.g. parsing.
---
### Author:  [@lungben](https://discourse.julialang.org/u/lungben)
#### Post date: [February 2, 2021, 11:23am UTC](https://discourse.julialang.org/t/converting-csv-string-values-to-floats-python-to-julia/54330/14 "2021-02-02T11:23:03Z")
You can use
> **[GitHub - lungben/TableIO.jl: A glue package for reading and writing tabular...](https://github.com/lungben/TableIO.jl)**
>
> A glue package for reading and writing tabular data. It aims to provide a uniform api for reading and writing tabular data from and to multiple sources. - GitHub - lungben/TableIO.jl: A glue packag...
to directly import Pandas-style HDF files in Julia - for HDF, it calls Pandas under the hood (with Pandas.jl), therefore it should support all Python data types (including e.g. pickled strings).