# Readlines(filepath) and read(filepath,String) read different content

**URL:** <https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777>\
**Category:** General Usage\
**Tags:** file\
**Created:** [October 1, 2024, 10:10am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777 "2024-10-01T10:10:22Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 1, 2024, 10:10am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/1 "2024-10-01T10:10:22Z")

</div>

Hey Julianners,  
This is fairly strange behavior as the `readlines(fila_path)` doesn’t read the last enter from the file. Why is that?

```julia
file_path="test/playground/test/src/profile_startup.jl"
lines = readlines(file_path, keep=false)
@show lines
show(join(lines, "\n"))
println()
show(read(file_path,String) )
println()
@assert join(lines, "\n") == read(file_path,String) 

```

If I remove the empty line from the file, then they read the same.  
Is this intentional? It caused a pretty big headache to find how did the file changed without any reason, due to there were different version used in the two place.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 1, 2024, 10:15am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/2 "2024-10-01T10:15:43Z")

</div>

using the:  
`lines = collect(eachline(open(file_path)))`  
Also causes ambiguous behavior.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 1, 2024, 10:20am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/3 "2024-10-01T10:20:20Z")

</div>

Ok, this is actually pretty strange. So I just moved forward with this solution:  
`lines=split(read(file_path,String), "\n")`

But maybe worth note that the `readlines` actually doesn’t read the last empty line from the file in certain case. And if there is no empty line then it read the last line… So it is pretty ambiguous.

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [October 1, 2024, 10:44am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/4 "2024-10-01T10:44:07Z")

</div>

There is a keyword parameter `keep` in `readlines`, which is default `false` and if `true` it keeps the new line in the strings. This may be correlated.

At least, the function `readlines` is not documented accurately, as `keep` is just indirectly documented in the examples. So I would think this is a bug or at least bad documentation. You should open an issue for `readlines`.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 1, 2024, 12:16pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/5 "2024-10-01T12:16:05Z")

</div>

I saw that flag, but that causes even more problem sadly.  
So with `keep=true` it actually parse somehow like each `\n` is equal with double `\n`. But the last `\n` equal with 1 `\n`. So in spite of it is not impossible to create a parsing for this to reconstruct the `\n` numbers accurately it is still pretty bad behavior.  
The `readlines` should return with everything that is in the file like the `read` does.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [October 1, 2024, 1:59pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/6 "2024-10-01T13:59:21Z")

</div>

It’s a little philosophical: what is a “line”? Is a file that contains just a `"\n"` one line or two? Does a line end with a `\n` or does `\n` split lines? How many lines does `"1\n2\n3\n"` have and is it the same as `"1\n2\n3"`?

Yes, `readlines` with the default keep argument treats files with a final `\n` as exactly the same as those without it. If you need to detangle the two, use `keep=true` and it’ll return everything that is in the file like the `read` does — just concatenate the outputs (without adding extra `\n`s) and you’ll get back the same thing as `read(_, String)`.

I think it’s good to play around a bit with these things to get an understanding of the behaviors:

```julia-repl
julia> readlines(IOBuffer("1\n2\n3"))
3-element Vector{String}:
 "1"
 "2"
 "3"

julia> readlines(IOBuffer("1\n2\n3\n"))
3-element Vector{String}:
 "1"
 "2"
 "3"

julia> readlines(IOBuffer("1\n2\n3"), keep=true)
3-element Vector{String}:
 "1\n"
 "2\n"
 "3"

julia> readlines(IOBuffer("1\n2\n3\n"), keep=true)
3-element Vector{String}:
 "1\n"
 "2\n"
 "3\n"

```

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [October 1, 2024, 2:18pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/7 "2024-10-01T14:18:14Z")

</div>

Yet, the intended behavior should be documented clearly.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 1, 2024, 3:35pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/8 "2024-10-01T15:35:22Z")

</div>

> readlines(IOBuffer(“1\n2\n3\n”))

Thank you for the answers!

Well a little bit philosophical indeed. I would be of course voting for the version what we see in the file view and also to not lose data during the transformation. So if I see an empty line in the end of the file in the file view then I would say there is a “line” in the end. But I understand that here we interpret it like there is a “new line” with nothing in it… so we eventually drop it when it is false.  
It would be nice if there would be a chance to vote for what is the standard. One objective point for the file view to get that extra emtpy `""` in the end is because we will actually lose information by `readlines` and then `join`ing back together. So it would be “ok” to be able to `readlines` and the `join` back together as many time as you want without losing information.  
So this:

```julia
julia> readlines(IOBuffer("1\n2\n3\n"))
4-element Vector{String}:
 "1"
 "2"
 "3"
 "" # this would be the way to go I think to not allow losing data

```

Anyways, It is just an opinion ofc.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 1, 2024, 3:37pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/9 "2024-10-01T15:37:48Z")

</div>

I mean… without reading the docs I would expect it to preserve every information in every case. 😃

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [October 1, 2024, 4:08pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/10 "2024-10-01T16:08:56Z")

</div>

> [@Marcell\_Havlik](#):
>
> what we see in the file view

What file view? Different tools display/treat files without a final `\n` (or `\r\n`) differently. There’s not really a standard; one such standard says it’s just [incomplete](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_195). GitHub doesn’t even show the line; instead it flags final lines that are _missing_ their newline. `wc` — a utility whose only purpose is to count — doesn’t count incomplete lines. `1\n2` only has one line in its view. Some editors even silently add missing final newlines upon save.

If you don’t want to “lose data” (including the newline flavor), use `keep=true`.

> [@Marcell\_Havlik](#):
>
> One objective point for the file view to get that extra emtpy `""` in the end is because we will actually lose information by `readlines` and then `join`ing back together.

But empty lines at the end of a file already mean something and you’d want to preserve them, too. `["1","2","3",""]` is what you get from `readlines(IOBuffer("1\n2\n3\n\n"))`.

The “inverse” of the default `readlines(x)` is not `join` with `\n`, but rather `foreach(println, _)`. And the “inverse” of `readlines(x; keep=true)` is `foreach(print, _)`. Yes, the default might add a final newline if your file doesn’t already have it. It might also change the line separator. It’s an opinionated take that matches how many tools work with files.

---

<div class="post-metadata">

**Author:** ![tecosaur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tecosaur/32/23206_2.png) [@tecosaur](https://discourse.julialang.org/u/tecosaur)\
**Post date:** [October 2, 2024, 5:01am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/11 "2024-10-02T05:01:00Z")

</div>

> [@mbauman](#):
>
> There’s not really a standard; one such standard says it’s just [incomplete](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_195)

I think it’s also worth scrolling up a bit for the [POSIX definition of a line](https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#tag_03_185), “A sequence of zero or more non-\<newline\> characters plus a terminating \<newline\> character”.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 2, 2024, 8:50am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/12 "2024-10-02T08:50:18Z")

</div>

> The “inverse” of the default `readlines(x)` is not `join` with `\n` , but rather `foreach(println, _)` .  
> For me:

```julia
julia> lines=readlines(IOBuffer("1\n2\n3\n"))
3-element Vector{String}:
 "1"
 "2"
 "3"

julia> foreach(println, lines)
1
2
3

julia> lines=readlines(IOBuffer("1\n2\n3"))
3-element Vector{String}:
 "1"
 "2"
 "3"

julia> foreach(println, lines)
1
2
3

julia> 

```

So it isn’t an inverse of it. as we actually lost data if we had enter or not in the last line during the process.

For me I am fine not using it, I understand how it works and I appreciate your ideas and also respect this behavior so I don’t want to change.

Of course I would be curious who want to lose information occasionally, (I guess 90% of the people not aware of this behavior and luckily they don’t get errors due to this). 😃

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [October 2, 2024, 1:45pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/13 "2024-10-02T13:45:03Z")

</div>

> [@Marcell\_Havlik](#):
>
> Of course I would be curious who want to lose information occasionally, (I guess 90% of the people not aware of this behavior and luckily they don’t get errors due to this). 😃

I think it’s more that final trailing newlines (or the lack thereof) aren’t used to convey meaningful information in the vast majority of use-cases, and it’s [generally considered best practice](https://stackoverflow.com/questions/729692/why-should-text-files-end-with-a-newline) to ensure that text files end with that final newline.

I’m sorry you got caught out here, it’s always frustrating when that happens.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 2, 2024, 2:40pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/14 "2024-10-02T14:40:40Z")

</div>

I notice the comment there:

> <https://stackoverflow.com/questions/729692/why-should-text-files-end-with-a-newline>

> Now, on non POSIX compliant systems (nowadays that’s mostly Windows), the point is moot: files don’t generally end with a newline, and the (informal) definition of a line might for instance be “text that is separated by newlines” (note the emphasis). This is entirely valid. However, for structured data (e.g. programming code) it makes parsing minimally more complicated: it generally means that parsers have to be rewritten.

Julia could have done with the (_informal_) “Windows” way, read the last incomplete line, as a full line. I’m thinking would it be really bad? Less surprising to most users? Another option would be to throw an error for incomplete lines as a compromize.

I think readlines, and eachline would mostly be used for text, I doubt anyone is going to be implementing `cat`, or work with binary data. I think you even can’t since kept isn’t the default (not do you want it as such). [You of course want the possibility to work with binary data too, somehow, and if changing Julia then not in a breaking way, and would throwing an exception be considered a breaking change?]

Would _most_ Windows editors also add a newline? I.e. avoid the issue, or how did OP actually get into this mess?

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [October 2, 2024, 2:52pm UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/15 "2024-10-02T14:52:39Z")

</div>

> [@Palli](#):
>
> Julia could have done with the (_informal_) “Windows” way, read the last incomplete line, as a full line

That’s what we do. The default is just to strip trailing newline character(s), whether they exist or not. If you care about the existence of newlines, `keep` them.

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [October 3, 2024, 9:52am UTC](https://discourse.julialang.org/t/readlines-filepath-and-read-filepath-string-read-different-content/120777/16 "2024-10-03T09:52:11Z")

</div>

Yeah most of the people won’t care for each line, so most of the time people will luckily avoid running into issue due to this behavior, I know.  
For me I checked if the file I read from previous run changed and in the check I used `readlines` and before that it was `read` which read the file as it is correctly. So it randomly throwed file changed singal… It was pretty hidden issue. XD