How to stop a file watching task in Julia 1.8? Possible regression?

New to Julia
1

Hello!

I am using BetterFileWatching.jl and the following code snippet:

watch_task = @async watch_folder(".") do event
    @info "Something changed!" event
end

sleep(5)

# stop watching the folder
schedule(watch_task, InterruptException(); error=true)

The task runs fine, but the schedule to stop it never works. I assume the authors have had success with this expression in past julia versions (since they wrote it works), but for 1.8 this does not work anymore in my case. Is this a regression or is there another way to stop the task?

Kind regards

2

Hi!

Can you specify what you see? In my case, with both Julia 1.7 and 1.8, running the schedule line produces Task (done) @ followed by the hexadecimal task id.

You could also try running !istaskdone(watch_task) && yield(watch_task) after the schedule to explicitly yield to the watch_task in case it’s not picked up by the scheduler for some reason.

3

Hi!

Yes it indeed produces “Task done + hex id” here as well, but it keeps outputting changes in the directory after the task supposedly has been stopped. This is of course incorrect behaviour. Running your snippet piece by piece:

!istaskdone(watch_task)
false

Which indicates that the scheduler is not picking it up. The yield command produces:

yield(watch_task)
ERROR: yield: Task not runnable
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:35
 [2] yield (repeats 2 times)
   @ .\task.jl:831 [inlined]
 [3] top-level scope
   @ REPL[3]:1

Running the snippet in one go simply outputs “false”.

I am a bit perplexed about the difficulty of stopping a simple task in Julia right now.

Kind regards

4

See yellow:

image
image740Ă—684 43.5 KB

https://docs.julialang.org/en/v1/base/parallel/

5

The error seems to happen in the “do” statement. The task will be “done” since it runs through the code, but the “do” statement, never experiences “closure”.

6

Honestly this seems like it might be a library problem i.e. a bug as you’ve mentioned in your issue.

The watch_folder code creates its own inner Task with @async, and any exception on that is handled by passing an InterruptException to the Deno task which should end the file watching. However, the InterruptException we pass in the code from the README is never passed on to this inner task, so this never happens. (Basically, there’s three layers of Tasks, and we’re able to pass an Interrupt only to the outer layer, which doesn’t get propagated to the inner two.) This is all based on my so-so understanding of Tasks though, so I’ve subscribed to the Issue on Github to see what the update is going to be.

For now, it seems like the library is pretty young and not highly used (only 3 stars on the project), so you’re probably better off using the FileWatching standard library.

1 Like
7

Thanks for the look at it!

I tried using the standard library file watching, but I simply couldn’t get it to work asynchrounosly. I would risk that if for example 4 files are created at once, then only one of them is registered - this is suboptimal.

Do you happen to know how one could write an async version doing the same thing, but without this bug?

I tried to do the straight forward, swapping “watch_folder” function to the standard library, but it simply would not accept the do syntax, which seems necessary for this async behaviour.

Kind regards

8

How about?

import FileWatching: watch_folder
function watch_folder(f, path::AbstractString)
     while true
         f(watch_folder(path))
     end
end

I was able to run your original example with that function instead.

watch_task = @async watch_folder(".") ...
1 Like
9

Thanks this helps a lot!

The problem is now that I don’t understand what is “changed”, “renamed” and “timeout” actually means in the “FileEvent” result.

Renamed and timeout should just be change of name and “fail if taken too long”, but I cannot figure out how to split “changed” into “created” and “modified” which BetterFileWatching.jl does really smartly and intuitively. I am almost there, so I hope someone perhaps understands this part better.

Still surprised that something I thought would be so easy to do is actually the most difficult part to code up…

Kind regards

10

I’m glad you found it helpful.

For me renamed was set both for renamed files as well as newly created files. timeout refers to whether an optional argument to watch_folder was exceeded. You don’t need the construct sleep...schedule(..;error).

Here is the documentation for the FileWatching module:
https://docs.julialang.org/en/v1/stdlib/FileWatching/#FileWatching.watch_folder

1 Like
11

Thank you very much!

You are indeed right that renaming = new file, even though it annoys me not being able to discern between modified name and actual new file. Still for now it is workable for me.

Thank you for your help and time - it lets me progress my general code now.

I do hope though that this becomes much better, because I find it so unbelievable that a language as Julia, which is good at so many things, actually struggles to properly close down subtasks and perform IO operations - thankfully you gave me the quick workaround, which is why I marked your answer as solution.

Kind regards

1 Like
12

I’ve stumbled across a similar problem but i found that the following code works:

function keep_watching_folder(f, path)
    while true
        try
            f(watch_folder(path))
        catch
            break
        end
    end
end

by just calling unwatch_folder(path) somewhere.
Here’s an example:

julia> t = @task keep_watching_folder(println, ".")                                                                                                                                                                                                                                                                        
Task (runnable) @0x00000002995a59f0

julia> schedule(t)                                                                                                                                                                                                                                                                                                         
Task (runnable) @0x00000002995a59f0

julia> istaskstarted(t)                                                                                                                                                                                                                                                                                                    
true

julia> istaskdone(t)                                                                                                                                                                                                                                                                                                       
false

julia> unwatch_folder(".")                                                                                                                                                                                                                                                                                                 

julia> istaskdone(t)                                                                                                                                                                                                                                                                                                       
true

julia> istaskfailed(t)                                                                                                                                                                                                                                                                                                     
false
1 Like
13

Uh, I will give that a try and get back to you!

Kind regards