I need this for testing, it looks like i have the situation that the OS is buffering the write operation to a file. So in the test for isfile this is true, but the content has not been fully written.
Iâm 99% sure that if you donât see the file content update, itâs not the OS but actually the user space thatâs buffering the content. OS buffering happens when write to the underlying media is delayed and shouldnât be observable when reading the file from the same host, and should be only observable when, say, if the connection (usb/network) to the media is interrupted before the buffer is flushed.
In another word, it wasnât the OSâs doing (or it would have just give you the buffered content). Itâs the user space doing (possibly libcâs) so the OS has no way to tell you anything about it. Youâll have to ask the userspace code that does the buffering, possibly by just flushing the output.
1 Like
Testing if thereâs anyone opening the file should be possible, assuming you have permission to query the openning process. I believe fuser, lsof etc use procfs on Linux. Iâm not aware of any library implementation and Iâm not sure how itâs implemented elsewhere. (I think windows may simply block you from openning it so you may not have this issue there?)
I agree with most of your post and the cases where i have seen (excessive) OS buffering it was in an server/VM/container environment. In this case there might be a user space problem, but unfortunately i cannot control the actual file operations, as this is inside a library and julia side is ccalling the respective functions.
To be more concrete, weâre talking about the cairo surface create, use and destroy logic youâll find in Cairo.jl/Cairo.jl at master ¡ JuliaGraphics/Cairo.jl ¡ GitHub which is applied to CairoScriptSurfaces in Cairo.jl/Cairo.jl at master ¡ JuliaGraphics/Cairo.jl ¡ GitHub.
We have the open issue of CairoScript should probably not use finalizers ¡ Issue #215 ¡ JuliaGraphics/Cairo.jl ¡ GitHub and Andreas Noack created the fix Cairo.jl/test/runtests.jl at master ¡ JuliaGraphics/Cairo.jl ¡ GitHub, as (you can see in 215) he claims, only GC.cg() triggered the cairo_destroy_surface that writes the file in full. I donât agree on this (finalizer) after visiting the cairo source code some time, a single call to destroy should be enough and actuall is on linux.
Seems not to be on OSX, and recently i borrowed a Macbook (i donât have continuos access to it) and have seen the situation, that files were available in the filesystem only after some seconds. So my new theory is, his observation is correct, however itâs not the GC.gc() triggering the destroy, but the effect is only happening after some time.
- It just doesnât really make sense that the OS will hold the change youâve told it without letting anyone see. Itâll likely even be bad for performance since the OS has to keep multiple copies of the file content.
- To really debug, you should log all the cairo calls. You can then either see whether you are calling the functions in the order you think you are or you can then create a C-repro. Doesnât seem like anyone have done that in that issue.
- From a 5min read of cairo code, the flush isnât controlled by the surface, it is controlled by the device which is the script in this case. For the constructor you use, the script is never explicitly dereferenced after the surface acquires a reference of it. It doesnât seems that cairo has the floating reference like gobject does so the flushing will still depend on when the script is freed.
to 2. Actually at the time of this discussion i did this (gdb of julia-debug, debug version, breakpointing cairo_x calls) on linux and it did look good (destroy â file available). I donât have access to OSX, so i trusted Andreasâ solution.
to 3. I thought for some time into the same direction: What if there is no connection between surface and device destroy? But then the libcairo docu reads: âDecreases the reference count on surface by one. If the result is zero, then surface and all associated resources are freedâ and i see the same behavior on e.g. CairoSVGSurface(filename which test nicely on linux and OSX without destroying the device.
But iâll look into it, again.
btw: That âtracingâ triggered my question here: Testing GC and finalizers
Maybe a last question here: Do you have a recommendation for 2 (log all calls, on linux)? I just failed trying to use ltrace on a julia process, i see the startup, but then subprocesses or threads are doing the magic.
The device is a seperate object you construct. It is obviously not associated with the surface.
I saw that but I have no idea what you are asking about. finalizers are just normal functions. You test it by running it. Either directly, or indirectly via finalize or gc.
This is a scripting (in the sense that thereâs no separate AOT compilation stage for the user) and you have full control over all the source code so just do a grep over the code?
I tend to disagree, as the sequence of creating is:
cairo_script_create for the device, cairo_script_surface_create based on the device - while i agree the julia code doesnât hold reference, libcairo does.
Cairoscript is a very specific part of libcairo which is mainly for debugging and not clearly documented. Itâs also mentioned in the documentation, that I/O for libcairo should be rather done with streams, then file access. If the file is only closed on destroy device, then referencing is needed, but actually surface finish sounds like the method.
Well, this indirectly is my point here. Iâd like to write a test in such a way, that the finalizer is called and i can test that it was called - without creating state or something global. On REPL i (obviously <1.0) had situation where i wasnât sure.
I guess you mean putting the whole Pkg.test inside code_lowered or code_llvm? I thought about that, but didnât make progress. I just have a longer experience in using gbd/strace/ltrace.
No. The whole point is that you didnât release the reference to the script explicitly.
Well, thereâs nothing special about indirect. Itâs a function with no return value so you can only observe side effects. Youâll have to figure out how to do that for your specific case. Julia does not provide a way to tell you everytime a finalizer is called (other than calling the finalizer) just like it doesnât tell you everytime a function without a return value is called.
No. I mean âgrep over the codeâ as in just edit the source code.