HTTP.jl async is slow compared to python+aiohttp

General Usage Performance
45

Eureka! I have finally solved this after checking the HTTP.jl documentation and determining that there is a way to configure it to use OpenSSL instead of MbedTLS. This can be done by passing in socket_type_tls=OpenSSL.SSLStream to the request. Once I did it now seems to be running as fast as python now over HTTPS:

urls = map(i->"https://mur-sst.s3.us-west-2.amazonaws.com/zarr-v1/analysed_sst/$i.1.0", 0:99);
julia> @time asyncmap(url->HTTP.request("GET", url, status_exception=false, connection_limit=25, socket_type_tls=OpenSSL.SSLStream).body, urls)
  7.531837 seconds (779.31 k allocations: 5.198 GiB, 0.49% compilation time)

A big thank you to everyone for helping me work this out!

23 Likes