How to optymaly to save strings in JLD ? What wrong?

General Usage
1

i have long vecotr od srings, dane[:,5]
12245198 -element Array{Any,1}:
“aa”
“ac”
“…”
“ab”
“aa”

saved by
writecsv(“file.txt”,string.(dane[:,5]))

is only ±33 000 KB

but saved by
`file = jldopen(“file.jld”,“w”)
close(file)
file = jldopen(“file.jld”,“r+”)

@time write(file,“user”,string.(dane[:,5]))
close(file)`

created file over 40 000 000 KB !!! What wrong ? How to optymaly this vector to save in JLD ? …as string.
Paul

2

Try JLD2? Also please quote code with ` and blocks of code with ```.

3

Also, if each element of that array

  1. contains only ASCII characters,
  2. has uniform length, or variable length that can be bounded by a small number,

then Matrix{UInt8} is also a viable choice.

If the strings repeat a lot, you could collect them and index into it. See for example
https://github.com/JuliaComputing/PooledArrays.jl

4

Yes, JLD2 make to lage file also!
Paul

5

What happens if you convert the array to actually be of type Vector{String} instead of Vector{Any}?

6

Thanks, convert(Array{String},a) with JLD working better!

BUT, if i Have vector “a” {Any} like below. In this case is npossible to make CONVERT(…

If i am using string.(a) the saved data are huge:/
What to do ?
Paul

julia> a
2-element Array{Any,1}:
123456
“ABCDEF”

julia> string.(a)
2-element Array{String,1}:
“123456”
“ABCDEF”

julia> convert(Array{String},a)
ERROR: MethodError: Cannot convert an object of type Int64 to an object of type String
This may have arisen from a call to the constructor String(…),
since type constructors fall back to convert methods.
in copy!(::Base.LinearFast, ::Array{String,1}, ::Base.LinearFast, ::Array{Any,1}) at .\abstractarray.jl:559
in convert(::Type{Array{String,N}}, ::Array{Any,1}) at .\array.jl:235

7

I have !
convert(Array{String},string.(a))
:slight_smile:

8

string.(a) doesn’t modify a, it returns a new array.

1 Like