# How to test if a file is existing and 'closed'?

**URL:** <https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045>\
**Category:** General Usage\
**Created:** [August 25, 2018, 3:59pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045 "2018-08-25T15:59:52Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [August 25, 2018, 3:59pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/1 "2018-08-25T15:59:52Z")

</div>

I need this for testing, it looks like i have the situation that the OS is buffering the write operation to a file. So in the test for isfile this is true, but the content has not been fully written.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 25, 2018, 4:11pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/2 "2018-08-25T16:11:22Z")

</div>

I’m 99% sure that if you don’t see the file content update, it’s not the OS but actually the user space that’s buffering the content. OS buffering happens when write to the underlying media is delayed and shouldn’t be observable when reading the file from the same host, and should be only observable when, say, if the connection (usb/network) to the media is interrupted before the buffer is flushed.

In another word, it wasn’t the OS’s doing (or it would have just give you the buffered content). It’s the user space doing (possibly libc’s) so the OS has no way to tell you anything about it. You’ll have to ask the userspace code that does the buffering, possibly by just flushing the output.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 25, 2018, 4:16pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/3 "2018-08-25T16:16:46Z")

</div>

Testing if there’s anyone opening the file should be possible, assuming you have permission to query the openning process. I believe `fuser`, `lsof` etc use `procfs` on Linux. I’m not aware of any library implementation and I’m not sure how it’s implemented elsewhere. (I think windows may simply block you from openning it so you may not have this issue there?)

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [August 25, 2018, 5:10pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/4 "2018-08-25T17:10:17Z")

</div>

I agree with most of your post and the cases where i have seen (excessive) OS buffering it was in an server/VM/container environment. In this case there might be a user space problem, but unfortunately i cannot control the actual file operations, as this is inside a library and julia side is ccalling the respective functions.

To be more concrete, we’re talking about the cairo surface create, use and destroy logic you’ll find in [Cairo.jl/Cairo.jl at master · JuliaGraphics/Cairo.jl · GitHub](https://github.com/JuliaGraphics/Cairo.jl/blob/master/src/Cairo.jl#L162-L225) which is applied to CairoScriptSurfaces in [Cairo.jl/Cairo.jl at master · JuliaGraphics/Cairo.jl · GitHub](https://github.com/JuliaGraphics/Cairo.jl/blob/master/src/Cairo.jl#L474-L479).

We have the open issue of [CairoScript should probably not use finalizers · Issue #215 · JuliaGraphics/Cairo.jl · GitHub](https://github.com/JuliaGraphics/Cairo.jl/issues/215) and Andreas Noack created the fix [Cairo.jl/test/runtests.jl at master · JuliaGraphics/Cairo.jl · GitHub](https://github.com/JuliaGraphics/Cairo.jl/blame/master/test/runtests.jl#L220-L226), as (you can see in 215) he claims, only GC.cg() triggered the cairo\_destroy\_surface that writes the file in full. I don’t agree on this (finalizer) after visiting the cairo source code some time, a single call to destroy should be enough and actuall is on linux.  
Seems not to be on OSX, and recently i borrowed a Macbook (i don’t have continuos access to it) and have seen the situation, that files were available in the filesystem only after some seconds. So my new theory is, his observation is correct, however it’s not the GC.gc() triggering the destroy, but the effect is only happening after some time.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 25, 2018, 5:35pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/5 "2018-08-25T17:35:42Z")

</div>

1. It just doesn’t really make sense that the OS will hold the change you’ve told it without letting anyone see. It’ll likely even be bad for performance since the OS has to keep multiple copies of the file content.
2. To really debug, you should log all the cairo calls. You can then either see whether you are calling the functions in the order you think you are or you can then create a C-repro. Doesn’t seem like anyone have done that in that issue.
3. From a 5min read of cairo code, the flush isn’t controlled by the surface, it is controlled by the device which is the script in this case. For the constructor you use, the script is never explicitly dereferenced after the surface acquires a reference of it. It doesn’t seems that cairo has the floating reference like gobject does so the flushing will still depend on when the script is freed.

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [August 25, 2018, 6:07pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/6 "2018-08-25T18:07:42Z")

</div>

to 2. Actually at the time of this discussion i did this (gdb of julia-debug, debug version, breakpointing cairo\_x calls) on linux and it did look good (destroy → file available). I don’t have access to OSX, so i trusted Andreas’ solution.  
to 3. I thought for some time into the same direction: What if there is no connection between surface and device destroy? But then the libcairo docu reads: “Decreases the reference count on _`surface`_ by one. If the result is zero, then _`surface`_ and all associated resources are freed” and i see the same behavior on e.g. CairoSVGSurface(filename which test nicely on linux and OSX without destroying the device.

But i’ll look into it, again.

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [August 25, 2018, 6:12pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/7 "2018-08-25T18:12:25Z")

</div>

btw: That ‘tracing’ triggered my question here: [Testing GC and finalizers](https://discourse.julialang.org/t/testing-gc-and-finalizers/13814)

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [August 25, 2018, 6:28pm UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/8 "2018-08-25T18:28:48Z")

</div>

Maybe a last question here: Do you have a recommendation for 2 (log all calls, on linux)? I just failed trying to use ltrace on a julia process, i see the startup, but then subprocesses or threads are doing the magic.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 26, 2018, 1:14am UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/9 "2018-08-26T01:14:27Z")

</div>

> [@lobingera](#):
>
> If the result is zero, then _`surface`_ and all associated resources are freed

The device is a seperate object you construct. It is obviously not associated with the surface.

> [@lobingera](#):
>
> That ‘tracing’ triggered my question here: [Testing GC and finalizers](https://discourse.julialang.org/t/testing-gc-and-finalizers/13814)

I saw that but I have no idea what you are asking about. finalizers are just normal functions. You test it by running it. Either directly, or indirectly via `finalize` or `gc`.

> [@lobingera](#):
>
> Do you have a recommendation for 2

This is a scripting (in the sense that there’s no separate AOT compilation stage for the user) and you have full control over all the source code so just do a grep over the code?

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [August 26, 2018, 8:59am UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/10 "2018-08-26T08:59:49Z")

</div>

> [@yuyichao](#):
>
> The device is a seperate object you construct. It is obviously not associated with the surface.

I tend to disagree, as the sequence of creating is:  
cairo\_script\_create for the device, cairo\_script\_surface\_create based on the device - while i agree the julia code doesn’t hold reference, libcairo does.  
Cairoscript is a very specific part of libcairo which is mainly for debugging and not clearly documented. It’s also mentioned in the documentation, that I/O for libcairo should be rather done with streams, then file access. If the file is only closed on destroy device, then referencing is needed, but actually surface finish sounds like the method.

> [@yuyichao](#):
>
> Either directly, or indirectly via `finalize` or `gc` .

Well, this indirectly is my point here. I’d like to write a test in such a way, that the finalizer is called and i can test that it was called - without creating state or something global. On REPL i (obviously \<1.0) had situation where i wasn’t sure.

> [@yuyichao](#):
>
> This is a scripting (in the sense that there’s no separate AOT compilation stage for the user) and you have full control over all the source code so just do a grep over the code?

I guess you mean putting the whole Pkg.test inside code\_lowered or code\_llvm? I thought about that, but didn’t make progress. I just have a longer experience in using gbd/strace/ltrace.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [August 26, 2018, 11:33am UTC](https://discourse.julialang.org/t/how-to-test-if-a-file-is-existing-and-closed/14045/11 "2018-08-26T11:33:54Z")

</div>

> [@lobingera](#):
>
> i agree the julia code doesn’t hold reference

No. The whole point is that you didn’t release the reference to the script explicitly.

> [@lobingera](#):
>
> I’d like to write a test in such a way, that the finalizer is called and i can test that it was called - without creating state or something global.

Well, there’s nothing special about indirect. It’s a function with no return value so you can only observe side effects. You’ll have to figure out how to do that for your specific case. Julia does not provide a way to tell you everytime a finalizer is called (other than calling the finalizer) just like it doesn’t tell you everytime a function without a return value is called.

> [@lobingera](#):
>
> I guess you mean putting the whole Pkg.test inside code\_lowered or code\_llvm?

No. I mean “grep over the code” as in just edit the source code.