# Is there a limit on the number of @async tasks that can be scheduled?

**URL:** <https://discourse.julialang.org/t/is-there-a-limit-on-the-number-of-async-tasks-that-can-be-scheduled/58992>\
**Category:** General Usage\
**Created:** [April 10, 2021, 4:40pm UTC](https://discourse.julialang.org/t/is-there-a-limit-on-the-number-of-async-tasks-that-can-be-scheduled/58992 "2021-04-10T16:40:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![bjkaria](https://avatars.discourse-cdn.com/v4/letter/b/ee7513/32.png) [@bjkaria](https://discourse.julialang.org/u/bjkaria)\
**Post date:** [April 10, 2021, 4:40pm UTC](https://discourse.julialang.org/t/is-there-a-limit-on-the-number-of-async-tasks-that-can-be-scheduled/58992/1 "2021-04-10T16:40:40Z")

</div>

Hi,

I am using the HTTP.jl module to create multiple websocket connections and process data from them with @async. In the following program I see only 8 tasks ever succeeding. All the connections are going to be up longterm and they serve data at varying rates.

Using Julia 1.6.0  
OSs tested: Windows 10 Home, WSL2, Ubuntu 20.04

**asynctest.jl**

```julia
using JSON3, HTTP

const REST_URL = "https://api.pro.coinbase.com"
const WS_URL = "wss://ws-feed.pro.coinbase.com"
const PRODUCTS_URL = REST_URL * "/products"

function get_products()
    products = HTTP.request("GET", PRODUCTS_URL)
    prod_list = JSON3.read(products.body)
    return prod_list
end

prod_list = get_products()
println("Num Symbols:", size(prod_list))

global num_messages = 0
global num_subscribed = 0
@sync for prod in prod_list
    sleep(1)
    symbol = prod.id
    println(symbol)
    println("num_subscribed:", num_subscribed)
    t = @async HTTP.WebSockets.open(WS_URL) do ws
        global num_messages
        global num_subscribed
        subscribe = """{"type":"subscribe", "product_ids":["$symbol"], "channels": ["full"]}""" 
        write(ws, subscribe)
        num_subscribed += 1
        println("async num_subscribed:", num_subscribed)
        while !eof(ws)
            num_messages += 1
            msg = readavailable(ws)
            if (num_messages % 10000) == 0
                println("num_messages:", num_messages)
            end
        end
    end
end

```

The following output is seen  
**output**

```julia
C:\testprogs\julia>julia asynctest.jl
Num Symbols:(162,)
EOS-BTC
num_subscribed:0
MATIC-USD
num_subscribed:0
async num_subscribed:1
async num_subscribed:2
SUSHI-USD
num_subscribed:2
async num_subscribed:3
LINK-USD
num_subscribed:3
async num_subscribed:4
XTZ-BTC
num_subscribed:4
async num_subscribed:5
BCH-GBP
num_subscribed:5
async num_subscribed:6
LRC-BTC
num_subscribed:6
async num_subscribed:7
REN-BTC
num_subscribed:7
async num_subscribed:8
BNT-BTC
num_subscribed:8
YFI-BTC
num_subscribed:8
COMP-BTC
num_subscribed:8
NMR-USD
num_subscribed:8

```

The async\_num\_subscribed never goes above 8. I am fairly new to julia and maybe i am making some elementary errors. Looking for some guidance.

Bharat

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [April 10, 2021, 5:40pm UTC](https://discourse.julialang.org/t/is-there-a-limit-on-the-number-of-async-tasks-that-can-be-scheduled/58992/2 "2021-04-10T17:40:50Z")

</div>

From [https://juliaweb.github.io/HTTP.jl/stable/public\_interface/#Requests](https://juliaweb.github.io/HTTP.jl/stable/public_interface/#Requests), the part where keyword arguments are described, there is the default limit:

> `connection_limit = 8` , number of concurrent connections to each host:port.

You can raise it in your code with e.g.

```julia
HTTP.WebSockets.open(WS_URL; connection_limit = 10)

```

But I have no idea what a reasonable max number could be.

---

<div class="post-metadata">

**Author:** ![bjkaria](https://avatars.discourse-cdn.com/v4/letter/b/ee7513/32.png) [@bjkaria](https://discourse.julialang.org/u/bjkaria)\
**Post date:** [April 10, 2021, 6:04pm UTC](https://discourse.julialang.org/t/is-there-a-limit-on-the-number-of-async-tasks-that-can-be-scheduled/58992/3 "2021-04-10T18:04:01Z")

</div>

Thanks for the quick response, that seems to be it. i raised the limit to 1024 (way beyond my requirements), no issue at all handling a few thousand messages/sec with fairly low cpu usage and ~30-50mbps transfer rate.

Bharat