# Help writing a timeout macro

**URL:** https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591
**Category:** General Usage
**Tags:** macros, task
**Created:** [October 21, 2018, 3:47am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591 "2018-10-21T03:47:37Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![lstagner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lstagner/32/448_2.png) [@lstagner](https://discourse.julialang.org/u/lstagner)
#### Post date: [October 21, 2018, 3:47am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/1 "2018-10-21T03:47:37Z")

</div>

I have a function that can either execute quickly or take a very long time. I want to kill the function after a certain amount of time so I can fall back on a more reliable method. I should be able to do this using a macro. The problem is I am a bit out of my depth. I took info from [here](https://www.reddit.com/r/Julia/comments/78it4f/execute_a_function_with_a_timeout_time_limit/) and [here](https://stackoverflow.com/questions/51264467/timeout-function-in-julia) and this is what I have so far

```julia
macro timeout(time,f)
    quote
        t = Task($f)
        schedule(t)
        Timer(x -> (istaskdone(t) || Base.throwto(t,InterruptException())),$time)
        t
    end
end

```

The idea being I can just do

```julia
a = @timeout 5 short_or_long(args)

```

but my implementation is woefully wrong. Could anyone help?

---

<div class="post-metadata">

### Author: ![samoconnor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samoconnor/32/1802_2.png) [@samoconnor](https://discourse.julialang.org/u/samoconnor)
#### Post date: [October 21, 2018, 4:18am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/2 "2018-10-21T04:18:01Z")

</div>

I don’t believe there is a way to interrupt a Julia task that does not explicitly `yield` control.  
[https://github.com/JuliaLang/julia/issues/6283](https://github.com/JuliaLang/julia/issues/6283)

In HTTP.jl we have a timeout task that closes the network connection after a timeout. This causes the main task to abort with an EOF error next time it tries to use the connection. [https://github.com/JuliaWeb/HTTP.jl/blob/master/src/TimeoutRequest.jl#L20-L27](https://github.com/JuliaWeb/HTTP.jl/blob/master/src/TimeoutRequest.jl#L20-L27)

---

<div class="post-metadata">

### Author: ![lstagner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lstagner/32/448_2.png) [@lstagner](https://discourse.julialang.org/u/lstagner)
#### Post date: [October 21, 2018, 5:56am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/3 "2018-10-21T05:56:57Z")

</div>

Forgive my ignorance, but what does it mean to not explicitly yield control. Does this mean that a task that _does_ explicitly yields control can be interrupted?

---

<div class="post-metadata">

### Author: ![samoconnor](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samoconnor/32/1802_2.png) [@samoconnor](https://discourse.julialang.org/u/samoconnor)
#### Post date: [October 21, 2018, 7:50am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/4 "2018-10-21T07:50:44Z")

</div>

The way I understand it, a Julia task is it’s own master. If it calls `yield` (or `sleep`, or `wait` or some other API that ends up calling one of those), then it explicitly hands control to the Julia runtime. At all other times, a Julia task just executes compiled machine code like a compiled C program. i.e. there is no supervisor, or interpreter, or master thread, or any opportunity for preempting a Julia task.

> <https://github.com/JuliaLang/julia/issues/25353#issuecomment-354879008>
>
> This is fine on the REPL, method returns after interrupting task:
> \`\`\`
> julia\> t… = @async begin
> while true
> sleep(5)
> end
> end
> Task (runnable) @0x00007f1abbdc8010
> 
> julia\> @assert !istaskdone(t)
> 
> julia\> Base.throwto(t, InterruptException())
> ERROR: InterruptException:
> Stacktrace:
> \[1\] try\_yieldto(::typeof(identity), ::Base.RefValue{Task}) at ./event.jl:208
> \[2\] yieldto at ./event.jl:194 \[inlined\]
> \[3\] yieldto at ./event.jl:193 \[inlined\]
> \[4\] throwto(::Task, ::Any) at ./event.jl:218
> \[5\] top-level scope
> 
> julia\>
> \`\`\`
> 
> But this never comes out, the target task should have exited though:
> \`\`\`
> julia\> t = @async begin
> while true
> try
> sleep(5)
> catch ex
> error("task interrupted")
> end
> end
> end
> Task (runnable) @0x00007fc07a416cb0
> 
> julia\> @assert !istaskdone(t)
> 
> julia\> Base.throwto(t, InterruptException())
> ERROR (unhandled task failure): task interrupted
> Stacktrace:
> \[1\] error at ./error.jl:33 \[inlined\]
> \[2\] macro expansion at ./REPL\[1\]:6 \[inlined\]
> \[3\] (::getfield(, Symbol("##1#2")))() at ./task.jl:348
> 
> ... needs to be interrupted or killed
> \`\`\`
> 
> And this too, here the target task would not have exited:
> \`\`\`
> julia\> t = @async begin
> while true
> try
> sleep(5)
> catch ex
> if isa(ex, InterruptException)
> println("handled this exception, resuming")
> else
> rethrow(ex)
> end
> end
> end
> end
> Task (runnable) @0x00007f0fea006cb0
> 
> julia\> @assert !istaskdone(t)
> 
> julia\> Base.throwto(t, InterruptException())
> handled this exception, resuming
> 
> ... needs to be interrupted or killed
> \`\`\`
> 
> When not in REPL, \`throwto\` does not return in all the above cases, but the behavior is slightly different. It still needs to be interrupted or killed. There also seems to be a race condition here which causes the \`Workqueue inconsistency\` error.
> \`\`\`
> $ cat \<\<EOF \> /tmp/y.jl
> \> t = @async begin
> \> while true
> \> sleep(5)
> \> end
> \> end
> \> 
> \> @assert !istaskdone(t)
> \> 
> \> Base.throwto(t, InterruptException())
> \> EOF
> $ julia /tmp/y.jl 
> 
> WARNING: Workqueue inconsistency detected: popfirst!(Workqueue).state != :queued
> ERROR (unhandled task failure): InterruptException:
> ^C
> signal (2): Interrupt
> in expression starting at /tmp/y.jl:13
> unknown function (ip: 0x7fc08e075498)
> uv\_\_epoll\_wait at /data/Work/julia/sources/julia/deps/srccache/libuv-d8ab1c6a33e77bf155facb54215dd8798e13825d/src/unix/linux-syscalls.c:321
> uv\_\_io\_poll at /data/Work/julia/sources/julia/deps/srccache/libuv-d8ab1c6a33e77bf155facb54215dd8798e13825d/src/unix/linux-core.c:267
> uv\_run at /data/Work/julia/sources/julia/deps/srccache/libuv-d8ab1c6a33e77bf155facb54215dd8798e13825d/src/unix/core.c:354
> process\_events at ./libuv.jl:82 \[inlined\]
> wait at ./event.jl:258
> task\_done\_hook at ./task.jl:257
> jl\_call\_fptr\_internal at /data/Work/julia/sources/julia/src/julia\_internal.h:380 \[inlined\]
> jl\_call\_method\_internal at /data/Work/julia/sources/julia/src/julia\_internal.h:399 \[inlined\]
> jl\_apply\_generic at /data/Work/julia/sources/julia/src/gf.c:2082
> jl\_apply at /data/Work/julia/sources/julia/src/julia.h:1474 \[inlined\]
> finish\_task at /data/Work/julia/sources/julia/src/task.c:233
> start\_task at /data/Work/julia/sources/julia/src/task.c:276
> unknown function (ip: 0xffffffffffffffff)
> unknown function (ip: 0xffffffffffffffff)
> Allocations: 826398 (Pool: 825762; Big: 636); GC: 1
> \`\`\`

> Does this mean that a task that _does_ explicitly yields control can be interrupted?

Yes

```julia
julia> t = @async try while true sleep(1) ; println("tick") ; end catch e println("stopped on $e") endtick
tick
tick
tick

julia> @async Base.throwto(t, EOFError())
stopped on EOFError()
Task (runnable) @0x000000011555b610

julia> t
Task (done) @0x000000011555b3d0

```

---

<div class="post-metadata">

### Author: ![captchanjack](https://avatars.discourse-cdn.com/v4/letter/c/74df32/32.png) [@captchanjack](https://discourse.julialang.org/u/captchanjack)
#### Post date: [June 7, 2021, 5:09pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/5 "2021-06-07T17:09:00Z")

</div>

I’ll just leave my approach here…

```julia
macro timeout(expr, seconds=-1, cb=(tsk) -> Base.throwto(tsk, InterruptException()))
    quote
        tsk = @task $expr
        schedule(tsk)

        if $seconds > -1
            Timer((timer) -> $cb(tsk), $seconds)
        end

        return fetch(tsk)
    end
end

julia> @timeout (sleep(3); println("done")) 3.1
done

julia> @timeout (sleep(3); println("done")) 3
ERROR: TaskFailedException:
InterruptException:
Stacktrace:
 [1] try_yieldto(::typeof(Base.ensure_rescheduled)) at ./task.jl:656
 [2] wait at ./task.jl:713 [inlined]
 [3] wait(::Base.GenericCondition{Base.Threads.SpinLock}) at ./condition.jl:106
 [4] _trywait(::Timer) at ./asyncevent.jl:110
 [5] wait at ./asyncevent.jl:128 [inlined]
 [6] sleep at ./asyncevent.jl:213 [inlined]
 [7] (::var"#29#31")() at ./task.jl:112
Stacktrace:
 [1] wait at ./task.jl:267 [inlined]
 [2] fetch(::Task) at ./task.jl:282
 [3] top-level scope at REPL[3]:10

```

---

<div class="post-metadata">

### Author: ![fredcallaway](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredcallaway/32/20304_2.png) [@fredcallaway](https://discourse.julialang.org/u/fredcallaway)
#### Post date: [June 21, 2021, 4:33pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/6 "2021-06-21T16:33:57Z")

</div>

Thanks, this was super helpful! I found two problems with this approach though. First, the `return` confusingly just terminates the expression; it doesn’t allow you to assign a variable to the output of the task (compare to `x = return 4`). Second, when you queue up multiple tasks in a row, the Timer from the old task is still active and will end up throwing the interrupt exception to the main program (not sure why that happens). Here’s my version, which fixes these issues (also switches the argument order)

```julia
macro timeout(seconds, expr)
    quote
        tsk = @task $expr
        schedule(tsk)
        Timer($seconds) do timer
            istaskdone(tsk) || Base.throwto(tsk, InterruptException())
        end
        fetch(tsk)
    end
end

x = @timeout 1 begin
    sleep(0.5)
    println("done")
    1
end
@assert x == 1

```

---

<div class="post-metadata">

### Author: ![hhaensel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hhaensel/32/1207_2.png) [@hhaensel](https://discourse.julialang.org/u/hhaensel)
#### Post date: [November 7, 2022, 2:23pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/7 "2022-11-07T14:23:04Z")

</div>

Thanks for this useful snippet.

I wanted something very similar but with the possibility of a default value in case of failure. So I came up with this:

```julia
macro timeout(seconds, expr, fail)
    quote
        tsk = @task $expr
        schedule(tsk)
        Timer($seconds) do timer
            istaskdone(tsk) || Base.throwto(tsk, InterruptException())
        end
        try
            fetch(tsk)
        catch _
            $fail
        end
    end
end

x = @timeout 1 begin
    sleep(1.1)
    println("done")
    1
end "failed"

```

---

<div class="post-metadata">

### Author: ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)
#### Post date: [November 24, 2022, 9:27am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/8 "2022-11-24T09:27:54Z")

</div>

this is amazing guys. thanks!

---

<div class="post-metadata">

### Author: ![ErwanMeunier](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erwanmeunier/32/47054_2.png) [@ErwanMeunier](https://discourse.julialang.org/u/ErwanMeunier)
#### Post date: [February 19, 2023, 4:55pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/9 "2023-02-19T16:55:51Z")

</div>

I have tried all day long to replicate @hhaensel’s snippet above considering some arguments to be given to the function. Since I know Julia’s `@task` doesn’t allow any function to be called with arguments, I have tried fixing the latter by calling the function with the variable. However, nothing happens.  
My main hypothesis is that `i` is interpreted by the macro as a symbol and not as a value. Is there a trick to overcome this problem in non-interactive mode?  
Here is my minimal (non)-working example:

```julia
function test()
    f(x) = begin 2*x ; println(2*x) ; sleep(1.) end
    #println("Testing f: f(3)")
    for i in 1:10
        @timeout 10 f(i) "fail"
        #println(i)
    end
end

```

I apologize for my poor English ^^’

---

<div class="post-metadata">

### Author: ![ErwanMeunier](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erwanmeunier/32/47054_2.png) [@ErwanMeunier](https://discourse.julialang.org/u/ErwanMeunier)
#### Post date: [February 20, 2023, 8:37am UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/10 "2023-02-20T08:37:02Z")

</div>

I dived more deeply in the `@timeout` macro by removing the `try catch_ end` statement and just leaving `fetch(tsk)` instead. Now, I get the following error:

```julia
julia> test()
ERROR: TaskFailedException
Stacktrace:
 [1] wait
   @ ./task.jl:345 [inlined]
 [2] fetch
   @ ./task.jl:360 [inlined]
 [3] test()
   @ Main ~/Documents/GIT/GravityMachine/src/solveSPAexactly/solveSPA.jl:28
 [4] top-level scope
   @ REPL[7]:1

    nested task error: UndefVarError: f not defined
    Stacktrace:
     [1] (::var"#170#173")()
       @ Main ./task.jl:134

```

When I bring `f` out of `test` (so that `f` becomes global), Julia yields the same error but with `i` instead of `f`:

```julia
nested task error: UndefVarError: i not defined
    Stacktrace:
     [1] (::var"#187#189")()
       @ Main ./task.jl:134

```

I feel that my problem is related to a misunderstanding of meta-programming. In my opinion I am trying to evaluate the code before parsing it (which is impossible?!).

---

<div class="post-metadata">

### Author: ![ErwanMeunier](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erwanmeunier/32/47054_2.png) [@ErwanMeunier](https://discourse.julialang.org/u/ErwanMeunier)
#### Post date: [February 27, 2023, 1:59pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/11 "2023-02-27T13:59:47Z")

</div>

It seems what i was looking for is the following **function** (and not a macro):

```julia
function timeout(f, arg, seconds, fail)
    tsk = @task f(arg...)
    schedule(tsk)
    Timer(seconds) do timer
        istaskdone(tsk) || Base.throwto(tsk, InterruptException())
    end
    try
        fetch(tsk)
    catch _;
        fail
    end
end

```

---

<div class="post-metadata">

### Author: ![louisponet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/louisponet/32/2070_2.png) [@louisponet](https://discourse.julialang.org/u/louisponet)
#### Post date: [February 27, 2023, 4:13pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/12 "2023-02-27T16:13:50Z")

</div>

I’ve ran into issues with this one when using it in tasks where the task mostly completes within seconds but sometimes takes minutes with good reason, i.e. having a long timeout.  
It would essentially flood the scheduler with many timers that were waiting to finish.

I changed it a bit to this, and it seems to not run into the same issues anymore:

```julia
macro timeout(seconds, expr, err_expr=:(nothing))
    esc(quote
        tsk__ = @task $expr
        schedule(tsk__)
        start_time__ = time()
        curt__ = time()
        Base.Timer(0.001, interval=0.001) do timer__
            if tsk__=== nothing || istaskdone(tsk__)
                close(timer__)
            else
                curt__ = time()
                if curt __- start_time__ > $seconds
                    Base.throwto(tsk__, InterruptException())
                end
            end
        end
        try
            fetch(tsk__)
        catch err__
            if err__.task.exception isa InterruptException
                RemoteHPC.log_error(RemoteHPC.StallException(err__))
                $err_expr
            else
                rethrow(err__.task.exception)
            end
        end
    end)
end

```

---

<div class="post-metadata">

### Author: ![ErwanMeunier](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/erwanmeunier/32/47054_2.png) [@ErwanMeunier](https://discourse.julialang.org/u/ErwanMeunier)
#### Post date: [March 26, 2023, 12:32pm UTC](https://discourse.julialang.org/t/help-writing-a-timeout-macro/16591/13 "2023-03-26T12:32:16Z")

</div>

I successfully reached the solution i was looking for by using the `esc(.)` function. Then, I get the following snippet:

```julia
macro timeout(seconds, expr, fail)
    quote
        tsk = @task $esc(expr)
        schedule(tsk)
        Timer($(esc(seconds))) do timer
            istaskdone(tsk) || Base.throwto(tsk, InterruptException())
        end
        try
            fetch(tsk)
        catch _
            $(esc(fail))
        end
    end
end

```
