Reading binary file in julia 1.0

New to Julia
1

The following function used to work fine in julia 0.6 for reading a file.
I am having a hard time trying to use it in julia 1.0 (I get the message: no method matching read(::IOStream, ::Type{Float32}, ::Int64))

function read_bin(filename, dims::N) where N<:Integer
    stream = open(filename, "r");
    data = read(stream, Float32, dims );
    close(stream);
    return data
end

any suggestions? thanks.

2

See: PSA: use Julia 0.7 if you are upgrading which says:

The 0.7 release exists exactly for this reason: your old code will work on 0.7 and give you a specific warning about how to fix your code.

Also, please quote your code: PSA: how to quote code with backticks

1 Like
3

Thanks a lot for replying!

Using julia 0.7 I got the following warning:
Warning: read(s::IO, t::Type, d1::Int, dims::Int...) is deprecated, use read!(s, Array{t}(undef, d1, dims...)) instead.

However, I am still doing something wrong. This time concerning types I am passing to read!..

Anyhow, thanks again!

4

If you can give more information about the new error you’re seeing, perhaps we can help with that as well.

5

I have the same problem. When I use the read_bin function, the read data is missing. What best way to read a binary file?

6

You can simply do

open(read, filename)

open creates an IO object. The first argument to open is a function which is evaluated on the IO object, in this case read which simply collects the contents of the stream as a Vector{UInt8}.

Alternatively, you can read using memory mapping with

using Mmap
Mmap.mmap(filename)

This will return a memory-mapped Vector{UInt8}.

1 Like
7

It could be that I’m doing it wrong. But it’s not working. Can I send the file to experiment?

8

Define “not working”, could you show me the stack trace? It works for me on any file I try it on.

9

The returned data is the type 0x00 for all array.

10

I’m read the same file in R using readBin function, but the returned data is diferent.

11

That doesn’t make sense to me. Are you sure it’s all zeros? Can you take the sum of the data and confirm that it’s zero? What is the file, what are you expecting to see?

The only thing I can think of is that R is expecting some sort of specialized format and is showing you data starting at some offset. The Julia functions will give you the entire file from the first bit to the last.

12

yes, in R i define byte and dimension. The datas are complex from the file .mlc .

13

That makes sense then, Julia is just giving you the complete buffer as a flat Vector. If you know the offset you want to start at, you can do, for example

Mmap.mmap(filename)[offset:end]

You can reshape the array (from Vector to any higher rank array) using reshape, you can see documentation on this function by doing ?reshape in the REPL.

(A word of warning, though mmap is desirable in many cases, you can wind up writing to the file if you are not careful. If you are very worried about this happening, use open(read, filename) instead.)

2 Likes
14

Mmap.mmap docs say it supports read only opening, but I think the call looks like Mmap.map(open(filename,"r")) in that case.