How to read in a list of integers into an array

Specific Domains Statistics
1

I have been a number of files, each contains one list of integers. I would just like to read in the contents into an array in Julia. Here is an example file:

http://paste.ubuntu.com/p/pxmwPXkNNk/

How can I do that?

I tried:

using DelimitedFiles
numbers = readdlm("test.txt")

but that gives something that isn’t right at all.

2

As is, numbers = include("test.txt") works.

1 Like
3

Wow. Thanks!

4

No seriously, don’t do that.

What is that format supposed to be? How is it produced?

If you know what format it is supposed to be in then use a reader for that format. If you don’t know then it seems to be json compatible and I’ll just read it as json. In any case, never use include for this.

1 Like
5

The lists look exactly like those produced from Python but I think they in fact came from the output of a piece of software called Gap. In any case, I need to read in the files as they are given to me. The main problem is just the carriage returns I think.

6

Well, readdlm clearly won’t work, and the new line isn’t the problem. You must first identify what format you are dealing with. If you don’t know or if the producer doesn’t specify, then as I said json should work. In any case, just don’t use include.

1 Like
7

I agree with this, even though I proposed the above “solution”. If you’re using this for anything more than a toy, do find a saner way to parse and do input verification.

9

I think one way in this case is to just input the entire thing as a single string and parse the string yourself. Probably not too bad because you can just split it into chunks by commas, discard the brackets, trim whitespace, and then parse to integer. Some regex wiz can probably do it in 2-3 lines of code.

10

I am open to any solution. I think the format is just [int, int, int, …, ] with possible extra white spaces and carriage returns.

11

I am not sure how I would read it as json. Could you give example code to do that?

12 
julia> using JSON

julia> write("data.json", "[1,2,3,4,5]");

julia> JSON.parsefile("data.json")
5-element Array{Any,1}:
 1
 2
 3
 4
 5

then convert the vector to whatever numeric type you want.

2 Likes
13

Thanks. Now I see that JSON.parsefile can’t cope with the large integers. It gives 0 for all the numbers which are too big. This is even before attempting to convert the vector. This file shows the problem Ubuntu Pastebin . Maybe I should ask a separate question for this aspect?

14

Edit: Nvm this answer, see answer below.

Ah, I didn’t look at the sample file and didn’t notice you needed that big numbers. It probably makes sense to write your own little parser function then like

function read_data(s) 
    joined = join(split(s))
    no_brackets = SubString(joined, nextind(joined, 1), prevind(joined, lastindex(joined)))
    return parse.(BigInt, split(no_brackets, ','))
end

julia> read_data(s)
150-element Array{BigInt,1}:
                                                4
                                               12
                                               32
                                               80
                                              192
                                              448
                                             1024
                                             2304
                                             5120
                                            11264
                                            24576
                                            53248
                                           114688
                                           245760
                                           524288
                                          1114112
                                          2359296
                                          4980736
                                         10485760
                                         22020096
                                         46137344
                                         96468992
                                        201326592
                                        419430400
                                        872415232
                                                ⋮
        10803965149739796214962143785958640713728
        21778071482940061661655974875633165533184
        43896425332801061786775324358698099277824
...
3 Likes
15

To use that function do I need to read the whole file in as a string somehow first?

16 
JSON.parsefile("filename"; inttype=BigInt)
5 Likes
17

Nice!

1 Like
18

This is funny that we all missed this! Thank you.