Reading data .mat files and merge them

General Usage
1

I have multiples files .mat that are organized at the same way. They look like that:

# file name : Case_1 
# Created by Octave 4.2.0
# matrix
# other comments
1 2 3 
4 5 6

# vector
0.2
0.3
0.4

And I’d like to get these values and group them in two other files. Let’s assume the second file it’s like that:

# file name : Case_2
# Created by Octave 4.2.0
# matrix
# other comments
7 8 9 
10 11 12

# vector
0.5
0.6
0.7

So, the final files would look like that :

# file name :Merge Matrix Case_1 Case_2
# Created by Octave 4.2.0
# matrix
# other comments
1 2 3 7 8 9 
4 5 6 10 11 12

AND this other one:

# file name :Merge Vector Case_1 Case_2
# Created by Octave 4.2.0
# vector
0.2
0.3
0.4
0.5
0.6
0.7

Anyone know if it’s really possible to do it in Julia ? Thanks in advance!
p.s : I have no access to the code source in octave, I only have the output data.

2 
using DelimitedFiles

function quick_and_dirty_read(filename)
    m = readdlm(filename; comments = true, comment_char = '#')
    vector_rows = isempty.(m[:, 2])
    Int.(m[.!vector_rows, :]), Float64.(m[ vector_rows, 1]) # modify types or do something else if necessary
end

reads the file, do the concatenation, then write it out similarly.

1 Like
3

Thanks for your answer. Unfortunately, I have tried to install this pack but I got this error :

julia> Pkg.add("DelimitedFiles")
ERROR: unknown package DelimitedFiles
macro expansion at .\pkg\entry.jl:53 [inlined]
(::Base.Pkg.Entry.##1#3{String,Base.Pkg.Types.VersionSet})() at .\task.jl:335

Do you have an idea? I actually use Juno for JuliaPro 0.6.2.2

4

I am guessing the answer was for 0.7/1.0. For 0.6, you don’t need to import anything for readdlm function.

2 Likes
5

that’s true thanks !!

6

Yes, since 1.0 is released I am now assuming that everyone uses that :wink:

1 Like
7

I have tried to change the type of my data, but I got this message:

ERROR: LoadError: MethodError: no method matching zero(::Type{Any})
Closest candidates are:
  zero(::Type{Union{Missings.Missing, T}}) where T at C:\JuliaPro-0.6.2.2\pkgs-0.6.2.2\v0.6\Missings\src\Missings
.jl:97
  zero(::Type{Base.LibGit2.GitHash}) at libgit2\oid.jl:106
  zero(::Type{Base.Pkg.Resolve.VersionWeights.VWPreBuildItem}) at pkg\resolve\versionweight.jl:82

My code is

for file in list_file
    data = readdlm(file; comments = true, comment_char = '#')
    vector_rows = isempty.(data[:, 2])
    float(data[vector_rows, :])  
end

I have also tried by using convert(Float64,data) but I got another error:

ERROR: LoadError: MethodError: Cannot `convert` an object of type Array{Any,2} to an object of type Float64
This may have arisen from a call to the constructor Float64(...),
since type constructors fall back to convert methods.

Do you have any idea how I could convert my data? Thanks in advance.

8

For the first error, it would be better to see the code that produced it.

For the second one, try float. (not the ., for broadcasting).

9

The error ERROR: LoadError: MethodError: no method matching zero(::Type{Any}) has been produced by the following code, using float():

clearconsole()
# this code aims to get the values of a .mat file, then save it in dataframe 
# each file represents a period, so the goal is to concanet the files in order to  have all the periods into a SAME dataframe 
# in order to respect the standard model, the lignes and columns might be transposed. 
using DataFrames
list_file = readdir() # getting file names
key = "1" 
nb_hours = 24  
nb_points = 23 

list_file_real = [file for file in list_file if contains(file,key)]  # excluding unnecessary files
periods= ["1-1_31-3","1-4_30-6","1-7_30-9","1-10_31-12"]
# Getting the real order
# WORKAROUND https://stackoverflow.com/questions/45013963/julia-sort-string-array-by-substring-containment
sort!(list_file_real,by=t->first(filter(x->contains(t,x[2]),enumerate([periods;""])))[1]) 
# the information contained in the following lines and columns:[nb_hours+1:nb_hours + nb_points,1 ] don't need to be repeted so
# they can placed OUT of the loop
file = list_file_real[1]
data_temp = readdlm(file; comments = true, comment_char = '#')
vector_final_case1_ProbPower = data_temp[nb_hours+1:nb_hours + nb_points,1]

# list of arrays that will keep the values of each file 
matrix_test_case1 = []
vector_average_test_case1 =[]

matrix_test_case2 = [] 
vector_average_test_case2 = []

for file in list_file_real
    data = readdlm(file; comments = true, comment_char = '#')
    vector_rows = isempty.(data[:, 2])
    float(data[vector_rows, :]) 
    # case 1 data 
    probabilities_case1_data =transpose(data[1:nb_hours,1:nb_points]) 
    average_power_case1_data = data[nb_hours + nb_points+1:nb_hours + nb_points+nb_hours,1]

    push!(matrix_test_case1, probabilities_case1_data)
    push!(vector_average_test_case1,average_power_case1_data)
    # case 2 data 
    probabilities_case2_data = data[nb_hours + nb_points+nb_hours + 1:nb_hours + nb_points+nb_hours+nb_hours,1:nb_points] 
    average_power_case2_data = data[nb_hours + nb_points+nb_hours+nb_hours+nb_points+1:nb_hours + nb_points+nb_hours+nb_hours+nb_points+nb_hours,1]
    # 
    push!(matrix_test_case2, probabilities_case2_data)
    push!(vector_average_test_case2, average_power_case2_data)

end