I haven’t been using Julia for a year now. And this morning I am struggling to save a two-dimensional array of complex numbers in a file in Julia so that I can open it up in Mathematica. What is the easiest way of doing this?
It looks like Mathematica supports reading from CSV files, so perhaps you can just do:
julia> using DelimitedFiles
julia> M = rand(ComplexF64, 3, 5)
3×5 Array{Complex{Float64},2}:
0.27002+0.970939im 0.153381+0.512542im 0.67822+0.371012im 0.272543+0.45637im 0.313847+0.138268im
0.104664+0.140601im 0.525328+0.617633im 0.712583+0.473099im 0.321276+0.854511im 0.883507+0.41321im
0.732629+0.0214171im 0.822763+0.360661im 0.791205+0.94645im 0.954784+0.0411716im 0.0109759+0.3415im
julia> writedlm("test.csv", M)
2 Likes
It’s not elegant, but for imperoperability’s sake saving the real and imaginary parts separately is pretty simple. You can get them with real.(A) and imag.(A) respectively. If you already have a way of getting real matrices into mathematica, you can use whatever that is on the real and imaginary parts separately, then simply add them in mathematica.
1 Like
The problem is in Mathematica complex numbers are denoted with “i” not im.
that could do. But seems very clumsy. Why would there not be a simpler solution. It bothers me. I don’t know about other scientific disciplines, but physics is filled with complex numbers (and so is EEE). So it seems odd that complex numbers get pushed to the side.
If anything, complex numbers are “pushed aside” by the IEEE standards on which computers operate, not by either of Julia or mathematica which both handle complex numbers quite well. Because there isn’t a standard binary (or unicode, for that matter) representation of a complex number, there isn’t really a standard format which allows Julia and mathematica to communicate with each other.
Therefore, unless you know of a standard data format that can be output and read by both, your only options are to have a custom output from Julia, a custom parser in mathematica, or both.
4 Likes
If that’s the only problem, you can postprocess the file and remove all m’s:
cat test.csv | tr -d 'm' > mathematica.csv
1 Like
I originally thought you’d either have to teach Julia “how to Mathematica”, or teach Mathematica “how to Julia”, since there’s no obvious common format. But perhaps they can meet halfway. You could do something like this in Julia:
using DelimitedFiles
M = rand(ComplexF64, 42, 42)
mcomplex(c) = string(real(c), ((imag(c) > 0) ? "+" : ""), imag(c), "*I")
writedlm("/tmp/test.csv", mcomplex.(M), ',')
julia-1.1> M[1, 1]
0.1625286359541478 + 0.0675447712375079im
julia-1.1> mcomplex(M[1, 1])
"0.1625286359541478+0.0675447712375079*I"
and then do something like this in Mathematica:
Using ToExpression was the only way I could convince Mathematica that the strings in the CSV file are to be interpreted as numbers. I’m hoping that this isn’t the easiest way to do it, though, because it does feel clumsy…
I could wrap the Import command with ToExpression[ Import[“test.csv”, “Data”]]. That’s not too difficult to do. Thanks very much. I think this solution will do the job for now.
1 Like
The root problem here is that there seems to be no standardized, widely used file format for interchange of complex-number data.
For CSV files, Matlab reads b+bi, Python reads a+bj, Julia reads both but also a+bim which is what it writes, Mathematica needs a+b*I followed by ToExpression, …
HDF5 files, inexplicably, don’t have a standard complex-number type, and neither does NetCDF.
.mat files do have a documented complex number format, but it would be stretch to call these a “standard”. Mathematical only reads version 4 and 5 .mat files, whereas Julia can only write version 7.3 .mat files, so that’s no help here. Python also only knows about earlier .mat formats. (Version 7.3 .mat files are actually HDF5, which is great except that the lack of a standard complex-number type in HDF5 means that generic HDF5 software like h5py won’t recognize the complex numbers as such.)
In the near term, it might be nice if MAT.jl could output Matlab v5 .mat files for interchange purposes. In the long term, I wish HDF5 would standardize a complex-number datatype, but it’s been 20 years and they haven’t gotten around to it yet so I wouldn’t hold my breath unless there is major external pressure (ideally accompanied by code PRs supplied to HDF5).
10 Likes
Has anyone thought about adding complex number support to HDF5.jl along the same lines as is done in h5py or is that out of scope for the package? The h5py implementation is here and here and doesn’t look too heavy. Compatibility between HDF5.jl and h5py for complex types would be convenient.
@jkleinh: I have written a h5py compatible version here:
https://github.com/MagneticParticleImaging/MPIFiles.jl/blob/master/src/Utils.jl#L5
and
here:
https://github.com/MagneticParticleImaging/MPIFiles.jl/blob/master/src/Utils.jl#L42
I have thought as well if we should just integrate this into HDF5.jl.
1 Like
I would be in favor of following h5py here and integrating this with HDF5.jl since it seems to be fairly basic and universally useful functionality that people coming from python would expect to just work. Also h5py has had this functionality for a while and doesn’t seem to have run into any major problems with their approach as far as I can tell.
Yes, we probably then should follow h5py as well in their way how the store boolean values. @jkleinh could you open an issue to HDF5.jl so that we can discuss this there? My read implementation should probably be adapted since readmmap is not type stable anymore but this is doable using the low-level interface (similar to what I do here https://github.com/MagneticResonanceImaging/MRIReco.jl/blob/master/src/IO/ISMRMRD.jl#L9)
@tobias.knopp it looks like there is already an issue open for this. I made a comment to gauge support for implementing this.
I’ve just now made a pull request to HDF5.jl implementing the h5py strategy for dealing with complex numbers. Interested in what people think. Hopefully this can be merged.
1 Like