# Reading complex text files with vectors

**URL:** <https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954>\
**Category:** General Usage\
**Tags:** question, io\
**Created:** [August 25, 2021, 3:11am UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954 "2021-08-25T03:11:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![scone](https://avatars.discourse-cdn.com/v4/letter/s/b487fb/32.png) [@scone](https://discourse.julialang.org/u/scone)\
**Post date:** [August 25, 2021, 3:11am UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/1 "2021-08-25T03:11:43Z")

</div>

Hi,

I am going to apologize ahead of time for this sort of general question, which is sort of similar others that have been asked. I am trying to read files the below, which are velocity vectors, probed at the locations listed in the very long header. My question is, is there an easy way to read these vectors with commas as vectors into an array or dataframe? More generally, what is the easiest way to store them so they are actually usable? If I have a vector of vectors and want to take a norm or a mean, how can I index them all at once?

This is my best attempt so far:

```julia
function readFoamVectors(filename, skipstart)
    #open it
    f = open(filename)
    #read the lines
    lines = readlines(f)
    
    #matrix to hold the final product
    data = Any[]

    #loop through the rows
    for i = skipstart:length(lines)
        rowdata = split(lines[i],r"\) \(|\(|\)")
        rowadd = Any[]
        append!(rowadd,parse(Float64,rowdata[1]))
        for j = 2:length(rowdata)-1
            append!(rowadd,[readdlm(IOBuffer(string(rowdata[j])))])
        end
        push!(data, rowadd')
    end

    return data
end

```

It has crashed my computer, and it seems generally sloppy and is too slow. Any ideas are great - thanks.

Here is some sample data, in reality, there might be 30 probes, but if I include a file like that it is too many characters. Typically the files are large enough to be about 400Mb in this case:

```julia
# Probe 0 (0 0.05 0.1)
# Probe 1 (-0.1 0.3 0)
# Probe 0 1
# Time
   0.001694915254 (-5.123201266 0.05644463619 0.247042423) (-5.181674475 -0.001740347889 0.0001904027387)
   0.001715438236 (-5.14262357 0.0563816763 0.2243838269) (-5.207857987 -0.002015886388 0.0003267140884)
    0.00173599556 (-5.133826491 0.05655950346 0.203092524) (-5.204641937 -0.002208989055 0.0004473137306)
   0.001756639258 (-5.134623229 0.05663423636

```

---

<div class="post-metadata">

**Author:** ![elbersb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elbersb/32/26568_2.png) [@elbersb](https://discourse.julialang.org/u/elbersb)\
**Post date:** [August 25, 2021, 6:37am UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/2 "2021-08-25T06:37:48Z")

</div>

It’s not quite clear to me what the desired result is. Are the lines starting with # part of the data? The last line seems to be missing a closing parentheses, right? Maybe you could show the data structure that you would like to obtain based on this dataset.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 25, 2021, 7:21am UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/3 "2021-08-25T07:21:34Z")

</div>

> [@scone](#):
>
> It has crashed my computer

Really? Normally one gets an error message and a stacktrace, so this may not be a Julia issue.

> [@scone](#):
>
> an easy way to read these vectors with commas

I don’t see a single comma in the example data you provided.

Please do put some effort into providing an example dataset and the expected result.

> [@Please read: make it easier to help you](https://discourse.julialang.org/t/please-read-make-it-easier-to-help-you/14757):
>
> Welcome to the Julia Discourse! We are enthusiastic about helping Julia programmers, both beginner and experienced. This public service announcement (PSA) outlines best practices when asking for help. Following these points makes it easier for us to help you and more likely you’ll get a prompt, useful answer. Keywords are highlighted to make it easier to refer to specific points. Choose a descriptive title that captures the key part of your question, eg “plots with multiple axes” instead of …

---

<div class="post-metadata">

**Author:** ![Mattriks](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mattriks/32/351_2.png) [@Mattriks](https://discourse.julialang.org/u/Mattriks)\
**Post date:** [August 25, 2021, 7:26am UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/4 "2021-08-25T07:26:53Z")

</div>

Your question seems similar to: [https://discourse.julialang.org/t/dataframes-csv-how-to-read-vectors-from-csv/](https://discourse.julialang.org/t/dataframes-csv-how-to-read-vectors-from-csv/)

---

<div class="post-metadata">

**Author:** ![hzgzh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hzgzh/32/6596_2.png) [@hzgzh](https://discourse.julialang.org/u/hzgzh)\
**Post date:** [August 25, 2021, 10:08am UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/5 "2021-08-25T10:08:48Z")

</div>

```julia
line = "0.001694915254 (-5.123201266 0.05644463619 0.247042423) (-5.181674475 -0.001740347889 0.0001904027387)"
x = split(line)
for i in 1:length(x)
x[i] = strip(x[i],['(',')']
end

```

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [August 25, 2021, 12:20pm UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/6 "2021-08-25T12:20:24Z")

</div>

@scone, in case it helps find a simple file reader below where input file has no headers/comments:

```julia
function readprobes(file::String, nprobes)
    data = Matrix{Array{Float64}}(undef,0,nprobes+1)
    open(file) do io
        while !eof(io)
            str = strip.(split(readline(io), ('(',')')))
            str = str[.!isempty.(str)] # removes empty elements
            v = []
            for i in 1:nprobes+1
                push!(v, parse.(Float64, split(str[i])))
            end
            data = vcat(data, permutedims(v))
        end
    end
    return data
end

file = raw"C:\..\filename.txt"

a = readprobes(file, 2) # 2 probes

3×3 Matrix{Any}:
 [0.00169492] [-5.1232, 0.0564446, 0.247042] [-5.18167, -0.00174035, 0.000190403]
 [0.00171544] [-5.14262, 0.0563817, 0.224384] [-5.20786, -0.00201589, 0.000326714]
 [0.001736] [-5.13383, 0.0565595, 0.203093] [-5.20464, -0.00220899, 0.000447314]

```

---

<div class="post-metadata">

**Author:** ![scone](https://avatars.discourse-cdn.com/v4/letter/s/b487fb/32.png) [@scone](https://discourse.julialang.org/u/scone)\
**Post date:** [August 25, 2021, 10:03pm UTC](https://discourse.julialang.org/t/reading-complex-text-files-with-vectors/66954/7 "2021-08-25T22:03:01Z")

</div>

Thanks Rafael! That is better than what I had! I think I said “commas” instead of “parentheses” earlier- the Tamas will have to forgive me for writing while tired, haha.