# Foolproof method for converting to Float64

**URL:** https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011
**Category:** New to Julia
**Tags:** dataframes, convert
**Created:** [May 28, 2021, 7:11pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011 "2021-05-28T19:11:30Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![mjanun](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mjanun](https://discourse.julialang.org/u/mjanun)
#### Post date: [May 28, 2021, 7:11pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/1 "2021-05-28T19:11:30Z")

</div>

I am reading some data from a CSV file, which is intended to be Float64. But it does not always parses as Float64- some times its type is Any, some times String, some times integer, etc.

To convert it to Float64, I use `parse.(Float64,DataFrame.colname)` but it does not always work. The data may have a currency symbol or a comma separator.

Imagine that the data in that particular column could be one of the format given below.

Is there a way of converting it to Float64 which always work? I am looking for a simple function which converts it in to Float64.

```julia
A1 = ["£131,222.23","£223,422"] # I can't use $ symbol here as it throws an error.
A2 = ["131,222.23","223,422"]
A3 = ["£131222.23","£223422"]
A4 = ["131222.23","223422"]
A5 = [131222.23,223422]

```

---

<div class="post-metadata">

### Author: ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)
#### Post date: [May 28, 2021, 7:20pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/2 "2021-05-28T19:20:35Z")

</div>

For the strings, remove the £ and comma characters before parsing. For the values that are already floats, don’t do anything. If you’re not sure which type you have, either redesign your system so that you always have the same type by telling CSV to read everything as strings, or dispatch on the type.

---

<div class="post-metadata">

### Author: ![mjanun](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mjanun](https://discourse.julialang.org/u/mjanun)
#### Post date: [May 28, 2021, 7:33pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/3 "2021-05-28T19:33:13Z")

</div>

The CSV file has several different types of fields. I am not sure if one can specify a particular type for one specified column. I gave an example of only a part of the data

---

<div class="post-metadata">

### Author: ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)
#### Post date: [May 28, 2021, 7:43pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/4 "2021-05-28T19:43:57Z")

</div>

You can

[https://csv.juliadata.org/stable/#Providing-Types](https://csv.juliadata.org/stable/#Providing-Types)

---

<div class="post-metadata">

### Author: ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)
#### Post date: [May 28, 2021, 7:44pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/5 "2021-05-28T19:44:48Z")

</div>

The problem here is an expectation that data that isn’t quite numerical (contains units, commas, etc.) should still be treated numerically. One could argue that certain patterns like these should be caught by the CSV reader; however, doing that pattern matching is expensive to do by default, and might not actually be intended (some users might want the currency symbol, or the value isn’t actually a number, it just looks like one). In such a case, it’s best to write your own conversion routine that strips out such characters (with `replace` or something similar) and then converts the resulting strings to `Float64`.

---

<div class="post-metadata">

### Author: ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [May 28, 2021, 7:55pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/6 "2021-05-28T19:55:22Z")

</div>

Small aside, it seems like you’re parsing currency into floating point - note that this is [usually not a good idea](https://stackoverflow.com/questions/3730019/why-not-use-double-or-float-to-represent-currency#3730040), since floating point is _not_ capable of accurately representing every decimal point value you may get and may introduce roundoff errors (which is the last thing you want when dealing with currency). I’d recommend modeling your currency explicitly (e.g. with a struct containing two fields, one the leading part and the other being the 2 digit fraction, with appropriate definition for +,-,\*,/ etc. - Another option would be to save whole cents instead, removing the need for fractional parts in the first place) and instructing CSV.jl to parse the relevant columns into your format instead.

In either case, you still have to canoicalize the to-be-parsed element.

---

<div class="post-metadata">

### Author: ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [May 28, 2021, 7:59pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/7 "2021-05-28T19:59:22Z")

</div>

Unless OP is working in high performance finance where these round-off errors matter, this advice is likely overkill.

---

<div class="post-metadata">

### Author: ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [May 28, 2021, 8:01pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/8 "2021-05-28T20:01:34Z")

</div>

Uhh… I hope my banks frontend doesn’t aggregate my bank statements using floating point, even though that’s not high performance finance 😬 It’s good practice, since it prevents hard to track bugs with the order things are summed in.

---

<div class="post-metadata">

### Author: ![mjanun](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mjanun](https://discourse.julialang.org/u/mjanun)
#### Post date: [May 28, 2021, 8:04pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/9 "2021-05-28T20:04:53Z")

</div>

The data is originally contained in an excel file, so it is in one of the formats that excel regards as numerical. This data is then copied over to a CSV file as it is without any changes. I think Excel limits floats to 15-16 decimal places so accuracy is already reduced

---

<div class="post-metadata">

### Author: ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)
#### Post date: [May 28, 2021, 8:10pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/10 "2021-05-28T20:10:36Z")

</div>

There is also [Tutorial · XLSX.jl](https://felipenoris.github.io/XLSX.jl/stable/tutorial/) if you want to skip the CSV step.

---

<div class="post-metadata">

### Author: ![ptoche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ptoche/32/23554_2.png) [@ptoche](https://discourse.julialang.org/u/ptoche)
#### Post date: [May 28, 2021, 8:56pm UTC](https://discourse.julialang.org/t/foolproof-method-for-converting-to-float64/62011/11 "2021-05-28T20:56:29Z")

</div>

You may want to read [this](https://docs.julialang.org/en/v1/manual/strings/#Regular-Expressions) part of the docs. Also (disclaimer: I haven’t tried it myself), [this](https://github.com/jkrumbiegel/ReadableRegex.jl) could be useful.
