# How to read hex columns in CSV files
**URL:** https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493
**Category:** General Usage
**Tags:** question
**Created:** [January 25, 2023, 9:13am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493 "2023-01-25T09:13:08Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 9:13am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/1 "2023-01-25T09:13:08Z")
I have the following code:
```julia
using DLMReader
ds = filereader(logfile_name, types = [Float64, Int, Int, Int, Int, Int, Int, Int, Int, Int, Int, Int, Int],
header=[:time, :RTR, :addr, :Flags, :DLC, :d1, :d2, :d3, :d4, :d5, :d6, :d7, :d8], warn=0)
```
which used to work, but now I got a .csv file where the third column is hex encoded and now it fails.
How can I read hex encoded columns of a .csv file?
A solution with CSV.jl would also be fine with me, I could write a script fix\_csv.jl that just converts the hex column into decimal integer values.
Example of a column of the .csv file I would be able to read:
```julia
6976.81202,1,5c1,2,8,67,91,35,0,0,0,0,0,2
```
The value “5c1” should be parsed as hex value into an integer column.
It looks as if “informats” could be used to fix this issue?
[https://sl-solution.github.io/DLMReader.jl/stable/man/tutorial\_basic/#Reading-a-csv-file](https://sl-solution.github.io/DLMReader.jl/stable/man/tutorial_basic/#Reading-a-csv-file)
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [January 25, 2023, 9:28am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/2 "2023-01-25T09:28:27Z")
> [@ufechner7](#):
>
> It looks as if “informats” could be used to fix this issue?
~~I would say so. Register an informat which puts “0x” in front of the values of column3.
See also: [Informats · DLMReader](https://sl-solution.github.io/DLMReader.jl/stable/man/informat/)
~~
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 9:59am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/3 "2023-01-25T09:59:07Z")
Not working. The following code:
```julia
using DLMReader
function hex2int!(str)
val = parse(Int64, str, base=16)
replace!(str, str=>repr(val))
return str
end
register_informat(hex2int!)
```
gives the error:
```julia
ERROR: LoadError: AssertionError: informat must return its input or a subset of its input
```
A decimal value is often longer than its hex representation, but it seams as if this function is not allowed to return a string that is longer than the original string.
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 25, 2023, 10:13am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/4 "2023-01-25T10:13:58Z")
When you say a CSV.jl solution is fine do you mean a solution that directly parses the hex values or simply one that doesn’t fail and give you strings you can convert? That seems to work out of the box:
```julia
julia> using CSV
julia> CSV.File(IOBuffer("6976.81,1,5c1,2,8"), header = false)
1-element CSV.File:
CSV.Row: (Column1 = 6976.81, Column2 = 1, Column3 = String3("5c1"), Column4 = 2, Column5 = 8)
```
(I’ve shortened your example a bit to make it more legible)
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 10:17am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/5 "2023-01-25T10:17:29Z")
Created a bug report: [Informats cannot parse hex values · Issue #14 · sl-solution/DLMReader.jl · GitHub](https://github.com/sl-solution/DLMReader.jl/issues/14)
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 10:24am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/6 "2023-01-25T10:24:58Z")
Parsing hex as string is easy, but how do I convert:
```julia
julia> ds.addr
1318767-element Vector{Union{Missing, String}}:
"682"
"5c1"
...
```
into a Vector of integers?
I can parse scalar values using `parse(Int64, "0a", base=16)`, but how do I apply such a function on a Vector?
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [January 25, 2023, 10:28am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/7 "2023-01-25T10:28:29Z")
> [@oheil](#):
>
> which puts “0x” in front of the values
~~This was my idea. Not parsing the value inside the informat. Like: ``` function hex(str) return "0x"*str end register_informat(hex) ``` Did you read https://sl-solution.github.io/DLMReader.jl/stable/man/informat/ ?~~
> ~~Users can define their own informats, which is basically a function with one positional argument. The function must accept a special mutable string and returns its modified value (or returns a subset of it).
> ~~
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 10:30am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/8 "2023-01-25T10:30:11Z")
Nice idea, but it doesn’t work:
```julia
julia> include("scripts/read_hex.jl")
ERROR: LoadError: AssertionError: informat must return its input or a subset of its input
Stacktrace:
[1] register_informat(f::Function; quiet::Bool, force::Bool)
@ DLMReader ~/.julia/packages/DLMReader/gB2Mj/src/informats.jl:24
[2] register_informat(f::Function)
@ DLMReader ~/.julia/packages/DLMReader/gB2Mj/src/informats.jl:23
[3] top-level scope
@ ~/repos/LogFiles/scripts/read_hex.jl:13
[4] include(fname::String)
@ Base.MainInclude ./client.jl:476
[5] top-level scope
@ REPL[3]:1
in expression starting at /home/ufechner/repos/LogFiles/scripts/read_hex.jl:13
```
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 25, 2023, 10:33am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/9 "2023-01-25T10:33:14Z")
One way:
```julia
julia> x
3-element Vector{Union{Missing, String}}:
"682"
"5c1"
missing
julia> passmissing(y -> parse(Int64, y, base = 16)).(x)
3-element Vector{Union{Missing, Int64}}:
1666
1473
missing
```
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 10:49am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/10 "2023-01-25T10:49:39Z")
In which package is “passmissing” defined?
---
### Author:  [@nilshg](https://discourse.julialang.org/u/nilshg)
#### Post date: [January 25, 2023, 10:53am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/11 "2023-01-25T10:53:29Z")
It’s in `Missings`, but is also rexported by `DataFrames` so if you read from a CSV into a DataFrame it should be available anyway.
It’s a simple function though which you could replace by this if you don’t want an extra dependency:
```julia
julia> (y -> ismissing(y) ? y : parse(Int64, y, base = 16)).(x)
3-element Vector{Union{Missing, Int64}}:
1666
1473
missing
```
---
### Author:  [@ufechner7](https://discourse.julialang.org/u/ufechner7)
#### Post date: [January 25, 2023, 11:06am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/12 "2023-01-25T11:06:25Z")
Thanks!
Now using this code:
```julia
function hex2int(str)
if ismissing(str) return missing end
return parse(Int64, str, base=16)
end
ds = filereader(logfile_name, types = [Float64, Int, String, Int, Int, Int, Int, Int, Int, Int, Int, Int, Int],
header=[:time, :RTR, :addr_hex, :Flags, :DLC, :d1, :d2, :d3, :d4, :d5, :d6, :d7, :d8], warn=0)
modify!(ds, :addr_hex => byrow(hex2int) => :addr)
select!(ds, Not(:addr_hex))
select!(ds, [1,2,13,4,5,6,7,8,9,10,11,12])
```
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [January 25, 2023, 11:12am UTC](https://discourse.julialang.org/t/how-to-read-hex-columns-in-csv-files/93493/13 "2023-01-25T11:12:25Z")
You are right. My idea doesn’t work.
Adding a “0x” in front seems to be not possible as it would make the string longer, which somehow isn’t allowed.