# Create an indexed line from a vector

**URL:** <https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783>\
**Category:** Performance\
**Created:** [May 27, 2024, 1:45pm UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783 "2024-05-27T13:45:16Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ju\_ska](https://avatars.discourse-cdn.com/v4/letter/j/ecd19e/32.png) [@Ju\_ska](https://discourse.julialang.org/u/Ju_ska)\
**Post date:** [May 27, 2024, 1:45pm UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/1 "2024-05-27T13:45:16Z")

</div>

Hi,

I would like to take a vector like:

```julia
julia> vector
5-element Vector{Float64}:
 90.51
 51.79
 84.81
 66.11
 56.68

```

and output

```julia
 "0"
 "1:90.51"
 "2:51.79"
 "3:84.81"
 "4:66.11"
 "5:56.68"

```

that will be written into a file (using writedlm) as:

```julia
 0 1:90.51 2:51.79 3:84.81 4:66.11 5:56.68

```

So I wrote and tested this:

```julia
using BenchmarkTools

vector=rand(.1:.01:100.99, 50000)

function to_svm(vec)
    SVM_line=["0"]
    index=0
    for value in vec
        index+=1
        push!(SVM_line, string(index) * ":" * string(value))
    end
    return SVM_line
end

function to_svm_v2(vec)
SVM_line=["0"]
    for iter in eachindex(vec)
        push!(SVM_line, string(iter) * ":" * string(vec[iter]))
    end
    return SVM_line
end

@benchmark line=to_svm(vector)
@benchmark line_v2=to_svm_v2(vector)

```

The two methods take the same time, but I’d like to know if there’s a way to go faster. I have to process a lot of vector → lines before writing to the file so this is really the part of the code that I’d like to optimize.

---

<div class="post-metadata">

**Author:** ![zweiglimmergneis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zweiglimmergneis/32/7892_2.png) [@zweiglimmergneis](https://discourse.julialang.org/u/zweiglimmergneis)\
**Post date:** [May 28, 2024, 10:29am UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/2 "2024-05-28T10:29:35Z")

</div>

This is shorter but not faster

```julia
function to_svm_c(v)
    vcat("0", [string(i) * ":" * string(v[i]) for i in eachindex(v)])
end

```

It would be nice to have a version of `string()`, where you can reuse the buffer. I don’t know, if such a version exists. `string(n::Int)` is faster than `string(f::Float64)`. If you need only fixed precision, this might be of interest.

---

<div class="post-metadata">

**Author:** ![zweiglimmergneis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zweiglimmergneis/32/7892_2.png) [@zweiglimmergneis](https://discourse.julialang.org/u/zweiglimmergneis)\
**Post date:** [May 28, 2024, 10:55am UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/3 "2024-05-28T10:55:53Z")

</div>

If you are dealing with a restricted range in your input vector, you could try something like this.

```julia
d_float = Dict([f => string(f) for f in .1:.01:100.99])

function to_svm_d(ds, v)
    vcat("0", [string(i) * ":" * d_float[v[i]] for i in eachindex(v)])
end

```

On my machine this takes half the time compared to `to_svm`.

---

<div class="post-metadata">

**Author:** ![Sevi](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@Sevi](https://discourse.julialang.org/u/Sevi)\
**Post date:** [May 28, 2024, 11:08am UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/4 "2024-05-28T11:08:06Z")

</div>

Do you produce the vector just to write it to a file or do you need it later in your code?

If it’s the former, you can just skip the creation of the vector and directly write the content to a file, that might save you a little bit of extra time and memory (but perhaps not a huge amount).

Something like this (based on @zweiglimmergneis suggestion)

```julia
function to_file(v, filename)
    open(filename, "w") do file
        writedlm(file, ("0"))
        writedlm(file, (string(i) * ":" * string(v[i]) for i in eachindex(v)))
    end
end

```

Alternatively, you could use `writedlm` without even creating the lines manually (although that seems to be slightly slower on my system):

```julia
function to_file_v2(v, filename)
    open(filename, "w") do file
        writedlm(file, ("0"))
        writedlm(file, pairs(v), ':')
    end
end

```

EDIT:

So far the fastest I could get is ditching `writedlm` altogether and just writing everthing manually after `join`ing the lines together… 😅

```julia
function to_file(v, filename)
    open(filename, "w") do file
        write(file, "0\n")
        write(file, join((string(i) * ":" * string(val) for (i,val) in pairs(v)), '\n'))
    end
end

```

* * *

Side note: to get accurate benchmark results, the function arguments are usually quoted, e.g. `@benchmark to_svm($vector)` ):  
[https://juliaci.github.io/BenchmarkTools.jl/stable/](https://juliaci.github.io/BenchmarkTools.jl/stable/)

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 28, 2024, 1:57pm UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/5 "2024-05-28T13:57:20Z")

</div>

> [@zweiglimmergneis](#):
>
> It would be nice to have a version of `string()`, where you can reuse the buffer.

`print` to an `IOBuffer`. (This is what `string` does internally anyway.) For even more re-use, you can make a non-destructive `String`-like view of an `IOBuffer` with [StringViews.jl](https://github.com/JuliaStrings/StringViews.jl).

> [@Sevi](#):
>
> So far the fastest I could get is ditching `writedlm` altogether and just writing everthing manually after `join`ing the lines together… 😅
> 
> ```julia
> function to_file(v, filename)
> open(filename, "w") do file
> write(file, "0\n")
> write(file, join((string(i) * ":" * string(val) for (i,val) in pairs(v)), '\n'))
> end
> end
> 
> ```

Should be better still to ditch the `string` and `join` calls entirely and just `print` directly to the file, e.g. something like:

```julia
function to_file(v, filename)
    open(filename, "w") do file
        println(file, 0)
        for (i, val) in pairs(v)
            println(file, i, ':', val)
        end
    end
end

```

(which is also more readable than the `join` version in my opinion). Note that I use [`print` instead of `write`](https://discourse.julialang.org/t/difference-between-write-and-print/6943/2) to output the text representations of `i` and `val` directly. You could also use `enumerate` instead of `pairs` to allow `v` to be an iterator rather than a vector (and to guarantee that the output indices start at `1`).

A version of this idea is also in the Julia [performance tips (“Avoid string interpolation for I/O”)](https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-string-interpolation-for-I/O): don’t construct an intermediate string just to write it to a file.

---

<div class="post-metadata">

**Author:** ![Sevi](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@Sevi](https://discourse.julialang.org/u/Sevi)\
**Post date:** [May 28, 2024, 3:13pm UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/6 "2024-05-28T15:13:58Z")

</div>

> [@stevengj](#):
>
> Should be better still to ditch the `string` and `join` calls entirely and just `print` directly to the file, e.g. something like:

That’s what I thought as well (I tried it with `write` which still requires a `string` conversion), but neither the `write` version, nor the `print` version beat the `join` version on my system.

These are the three variants and their timings on a 2021 Macbook:

> **Code**
>
> ```julia
> using BenchmarkTools
> 
> vector=rand(.1:.01:100.99, 50000)
> 
> function to_file_print(v, filename)
> open(filename, "w") do file
> print(file, "0\n")
> for (i, val) in pairs(v)
> println(file, i, ':', val)
> end
> end
> end
> 
> function to_file_write(v, filename)
> open(filename, "w") do file
> write(file, "0\n")
> for (i, val) in pairs(v)
> write(file, string(i), ':', string(val), '\n')
> end
> end
> end
> 
> function to_file_join(v, filename)
> open(filename, "w") do file
> write(file, "0\n")
> write(file, join((string(i) * ":" * string(val) for (i,val) in pairs(v)), '\n'))
> write(file, '\n')
> end
> end
> 
> @benchmark to_file_print($vector, $"saved_vector_print.txt")
> @benchmark to_file_write($vector, $"saved_vector_write.txt")
> @benchmark to_file_join($vector, $"saved_vector_join.txt")
> 
> # outputs are identical
> read("saved_vector_print.txt", String) == read("saved_vector_write.txt", String) == read("saved_vector_join.txt", String) # true
> 
> ```

> **Timings**
>
> ```julia-repl
> julia> @benchmark to_file_print($vector, $"saved_vector_print.txt")
> BenchmarkTools.Trial: 217 samples with 1 evaluation.
> Range (min … max): 20.115 ms … 106.253 ms ┊ GC (min … max): 0.00% … 1.06%
> Time (median): 22.413 ms ┊ GC (median): 3.57%
> Time (mean ± σ): 23.078 ms ± 5.968 ms ┊ GC (mean ± σ): 2.68% ± 2.38%
> 
> ▂▂▄▄█▂▄▅▃                                                
> ▃▄▄▄▆██████████▇▆▇▆▆▄▃▃▃▃▃▃▁▁▃▁▁▁▁▁▁▁▃▁▁▁▁▃▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▃ ▃
> 20.1 ms Histogram: frequency by time 32.6 ms <
> 
> Memory estimate: 26.31 MiB, allocs estimate: 299501.
> 
> julia> @benchmark to_file_write($vector, $"saved_vector_write.txt")
> BenchmarkTools.Trial: 244 samples with 1 evaluation.
> Range (min … max): 17.986 ms … 41.734 ms ┊ GC (min … max): 0.00% … 3.22%
> Time (median): 19.924 ms ┊ GC (median): 4.89%
> Time (mean ± σ): 20.518 ms ± 2.952 ms ┊ GC (mean ± σ): 3.26% ± 2.87%
> 
> ▁▃▅▂█▆▃▂                                                    
> ▄█████████▅▄▄▃▃▁▃▁▃▁▂▁▁▁▂▂▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▂▁▁▁▁▁▁▁▁▁▂ ▃
> 18 ms Histogram: frequency by time 37.5 ms <
> 
> Memory estimate: 24.80 MiB, allocs estimate: 200012.
> 
> julia> @benchmark to_file_join($vector, $"saved_vector_join.txt")
> BenchmarkTools.Trial: 369 samples with 1 evaluation.
> Range (min … max): 10.134 ms … 70.317 ms ┊ GC (min … max): 0.00% … 11.10%
> Time (median): 12.996 ms ┊ GC (median): 10.07%
> Time (mean ± σ): 13.548 ms ± 4.263 ms ┊ GC (mean ± σ): 7.31% ± 5.83%
> 
> ▁▁▂▃▁█▆▆▂▂                                               
> ▃▃▅▆██████████▇▅▄▅▅▃▃▃▂▃▂▂▃▁▂▁▁▁▁▁▂▁▁▁▁▂▂▁▁▁▁▁▁▁▂▂▁▁▁▁▁▁▁▁▂ ▃
> 10.1 ms Histogram: frequency by time 26.6 ms <
> 
> Memory estimate: 26.88 MiB, allocs estimate: 250030.
> 
> ```

> **Versions**
>
> BenchmarkTools v1.5.0
> 
> ```julia-repl
> julia> versioninfo()
> Julia Version 1.10.3
> Commit 0b4590a5507 (2024-04-30 10:59 UTC)
> Build Info:
> Official https://julialang.org/ release
> Platform Info:
> OS: macOS (x86_64-apple-darwin22.4.0)
> CPU: 8 × Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
> WORD_SIZE: 64
> LIBM: libopenlibm
> LLVM: libLLVM-15.0.7 (ORCJIT, icelake-client)
> Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
> 
> ```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 28, 2024, 5:43pm UTC](https://discourse.julialang.org/t/create-an-indexed-line-from-a-vector/114783/7 "2024-05-28T17:43:11Z")

</div>

> [@Sevi](#):
>
> That’s what I thought as well (I tried it with `write` which still requires a `string` conversion), but neither the `write` version, nor the `print` version beat the `join` version on my system.

Maybe it’s just the lack of buffering for small writes? Which should be fixed by wrapping the file object in a buffered stream from [BufferedStreams.jl](https://github.com/JuliaIO/BufferedStreams.jl)? Just set

```julia
bfile = BufferedOutputStream(file)

```

and pass `bfile` instead of `file` to the `print` or `write` functions. (You might also need `close(bfile)` at the end of the `do` block?)