# Goose Attack against a Genie App

**URL:** <https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541>\
**Category:** Web Stack\
**Tags:** genie\
**Created:** [June 25, 2021, 10:36am UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541 "2021-06-25T10:36:18Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 10:36am UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/1 "2021-06-25T10:36:18Z")

</div>

I made a small Genie App, and I created a Goose Attack with Rust that tests it. I used Julia 1.6.1.

The app is essentially (which is routes.jl)

```julia
using Genie.Router, Genie.Renderer, Genie.Exceptions, JSON3

route("/") do
  serve_static_file("welcome.html")
end

route("/greeting", named = :greeting) do
  "Welcome to Genie! This site is served by Julia $(Base.VERSION) with $(Threads.nthreads()) threads."
end

route("/random", named = :random) do
 num = 1
 if haskey(@params, :num)
   num = parse(Int64, @params(:num))
 end
 JSON3.write(rand(Int, num))
end

```

I’m getting many errors like this in Genie

```julia
┌ Error: 2021-06-25 13:22:17 Base.IOError("stream is closed or unusable", 0)
└ @ Genie.AppServer ~/.julia/packages/Genie/Uvtzb/src/AppServer.jl:93

```

The Rust app gets timeouts.

The code for both is at [GitHub - StatisticalMice/performance-tests: Web based performance tests](https://github.com/StatisticalMice/performance-tests)

The full HTML report is here [GitHub & BitBucket HTML Preview](https://htmlpreview.github.io/?https://github.com/StatisticalMice/performance-tests/blob/main/GooseAttack/attack_report.html)

 ![Screenshot 2021-06-25 at 13.31.17](https://global.discourse-cdn.com/julialang/original/3X/4/8/48debe17876485dafa1015c31ec7a9c91458f84e.png)

Comments? Suggestions on improvement?

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 10:40am UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/2 "2021-06-25T10:40:02Z")

</div>

> **[Goose Attack: A Locust-inspired Load Testing Tool In Rust](https://www.tag1consulting.com/blog/goose-locust-inspired-load-testing-tool-rust)**
>
> Load testing is one of the tools we leverage regularly at Tag1. It can help prevent website outages, stress test code changes, and identify bottlenecks. The ability to run the same test repeatedly gives critical insight into the impact of changes to...

---

<div class="post-metadata">

**Author:** ![pfitzseb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pfitzseb/32/45566_2.png) [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)\
**Post date:** [June 25, 2021, 12:30pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/3 "2021-06-25T12:30:29Z")

</div>

FWIW, a simple HTTP server (code below) yields

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/3/13be63471b0fa8e444a73ab09fd08cbd0924f745.png)

for me, which doesn’t seem too bad.

The biggest difference is disabling logging of those “stream is closed or unusable” errors with e.g.

```julia
Logging.global_logger(NullLogger())

```

As a comparison: “normal” average request time in Genie is 47ms on my machine, whereas disabling the global logger gets it down to 21ms. The HTTP.jl server with logging took something like 35ms on average, IIRC. Not sure where the disparity comes from, tbh, but maybe Genie has some internal log handling.

> **HTTP server**
>
> ```julia
> using HTTP, Sockets, JSON, Logging
> using HTTP.URIs: queryparams, URI
> 
> const ROUTER = HTTP.Router()
> 
> function serveWelcome(req::HTTP.Request)
> return HTTP.Response(200, read(joinpath(@ __DIR__ , "welcome.html"), String))
> end
> function serveRandom(req::HTTP.Request)
> num = parse(Int, get(queryparams(URI(req.target)), "num", "0"))
> 
> return HTTP.Response(200, JSON.json(rand(Int, num)))
> end
> 
> HTTP.@register(ROUTER, "GET", "/", serveWelcome)
> HTTP.@register(ROUTER, "GET", "/random", serveRandom)
> 
> Logging.global_logger(NullLogger())
> HTTP.serve(ROUTER, Sockets.localhost, 8000)
> 
> ```

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 1:25pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/4 "2021-06-25T13:25:25Z")

</div>

> [@pfitzseb](#):
>
> `Logging.global_logger(NullLogger())`

I put this at the end of the `routes.jl`, the log statements went away, but the performance didn’t increase.

---

<div class="post-metadata">

**Author:** ![pfitzseb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pfitzseb/32/45566_2.png) [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)\
**Post date:** [June 25, 2021, 1:34pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/5 "2021-06-25T13:34:03Z")

</div>

I just disabled the logger in the Genie REPL. Not sure why that would be different, but then again this is the first time I’ve used Genie.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 1:52pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/6 "2021-06-25T13:52:59Z")

</div>

I tried to put this HTTP server code in the root of my repo, and serve content from GenieApp/public(/welcome.html). I can get it to serve that html page, but all content it tries to load is broken.

---

<div class="post-metadata">

**Author:** ![pfitzseb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pfitzseb/32/45566_2.png) [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)\
**Post date:** [June 25, 2021, 2:03pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/7 "2021-06-25T14:03:50Z")

</div>

Sure, that’s not what it is set up for. If you want a static file server (+ that one `/random` route), use something like

```julia
using HTTP, Sockets, JSON, Logging
using HTTP.URIs: queryparams, URI

const ROUTER = HTTP.Router()

function serveStatic(req::HTTP.Request)
    return HTTP.Response(200, read(joinpath(@ __DIR__ , "public", split(URI(req.target).path, '/')...), String))
end
function serveRandom(req::HTTP.Request)
    num = parse(Int, get(queryparams(URI(req.target)), "num", "0"))

    return HTTP.Response(200, JSON.json(rand(Int, num)))
end

HTTP.@register(ROUTER, "GET", "/random", serveRandom)
HTTP.@register(ROUTER, "GET", "/*", serveStatic)

Logging.global_logger(NullLogger())

HTTP.serve(ROUTER, Sockets.localhost, 8000)

```

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 2:15pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/8 "2021-06-25T14:15:05Z")

</div>

I’m using a different version:

```julia
using HTTP, Sockets, JSON3, Logging
using HTTP.URIs: queryparams, URI

const ROUTER = HTTP.Router()

function serveWelcome(req::HTTP.Request)
    return HTTP.Response(200, "Welcome to Genie! This site is served by Julia $(Base.VERSION) with $(Threads.nthreads()) threads.")
end

function serveRandom(req::HTTP.Request)
    num = parse(Int, get(queryparams(URI(req.target)), "num", "1"))

    return HTTP.Response(200, JSON3.write(rand(Int, num)))
end

HTTP.@register(ROUTER, "GET", "/", serveWelcome)
HTTP.@register(ROUTER, "GET", "/random", serveRandom)

Logging.global_logger(NullLogger())
HTTP.serve(ROUTER, Sockets.localhost, 8000)

```

It’s still not working properly, though. I’m running on a Mac, which might affect things.  
It’s in git.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 2:22pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/9 "2021-06-25T14:22:17Z")

</div>

The performance I get with this HTTP.jl version with code I have in git is

 ![Screenshot 2021-06-25 at 17.21.19](https://global.discourse-cdn.com/julialang/original/3X/6/9/69ad87bd8841129fad6828c0eec1e755cb936d4f.png)

---

<div class="post-metadata">

**Author:** ![pfitzseb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pfitzseb/32/45566_2.png) [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)\
**Post date:** [June 25, 2021, 2:57pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/10 "2021-06-25T14:57:18Z")

</div>

Ok, that looks much better, no? If you disregard the 99th and 100th percentile (or compile your code by hitting each route manually before starting the benchmark) then you get pretty good times.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 3:23pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/11 "2021-06-25T15:23:49Z")

</div>

The normal speed is better, but I still get **a lot** of errors during execution.

I added a Python Flask app to the git repo. It also produces connection errors on the Rust side. This makes me think that this is a problem with my Mac.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 3:52pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/12 "2021-06-25T15:52:56Z")

</div>

Most curious. The Python Flask app stopped producing any errors when I totally turned off the Mullvad VPN client. This didn’t help the two Julia apps, which still produce errors.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 3:57pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/13 "2021-06-25T15:57:15Z")

</div>

The test is running 30 concurrent users, which should not be too much for either my Mac or Julia, so something odd is happening.

If I use one client, there are no errors visible to goose connecting to Julia, but if I use two, there are.

I think it’s quite possible that the fast speed is actually too fast for the Mac.

---

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [June 25, 2021, 7:31pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/14 "2021-06-25T19:31:46Z")

</div>

IME, a single “hello world” response in a Genie app takes 15 ms compared to 5ms for both Flask and HTTP.jl. I don’t know why.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 25, 2021, 8:02pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/15 "2021-06-25T20:02:48Z")

</div>

Great stuff, thanks for setting these up and sharing!

Can you please try Genie v2 (currently on #master) it should be faster.

I’m also just cloned the repo and running the tests locally. But I’m curious what it looks like on your machine.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 9:10pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/16 "2021-06-25T21:10:24Z")

</div>

The problem **was** Mac. I re-ran the tests on Google GCP on a `e2-standard-4 (4 vCPUs, 16 GB memory)`. The VM is running Debian 10.  
Flask:

 ![Screenshot 2021-06-26 at 0.07.27](https://global.discourse-cdn.com/julialang/original/3X/6/e/6ed3118ca742af36d3089f56cc38a3cc2c7e34f2.png)  
Genie.jl:  
 ![Screenshot 2021-06-26 at 0.08.30](https://global.discourse-cdn.com/julialang/original/3X/9/5/95330ab6b28a6bc4307ea57646ba5d2e0f831033.png)  
HTTP.jl:  
 ![Screenshot 2021-06-26 at 0.09.40](https://global.discourse-cdn.com/julialang/original/3X/5/8/58b2e0c20656217ceda93383e286d7c846b714db.png)

The error messages from Genie were still coming until I turned off all logging. The test was run without logging.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 9:16pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/17 "2021-06-25T21:16:32Z")

</div>

Flask is actually doing pretty well with 52 ms average time. (It can probably be optimised further.)

HTTP.jl wins with roughly 6.6 ms average against Genie.jl’s 22 ms.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 25, 2021, 9:47pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/18 "2021-06-25T21:47:53Z")

</div>

With Genie 1.18 configured for production env (no Revise, no logging, etc) the timing on my machine is about identical with the timing for HTTP.jl (but both much worse than the ones on your server, I’m running on a Windows laptop):

Genie prod:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/9/2/9278310f423d4330fc11c3ff5105defe035698f5.png)

HTTP.jl

 ![image](https://global.discourse-cdn.com/julialang/original/3X/3/e/3e53739d101a0efd1713196f9e40f01bf342aab8.png)

Can you please try my Genie fork configured for prod on your server?

> <https://github.com/essenciary/performance-tests/tree/main/GenieApp>
>
> //github.com/essenciary/performance-tests/tree/main/GenieApp

* * *

I’m not sure about the errors, seem to be coming from Rust, Goose outputs a lot of:

```julia
21:58:29 [WARN] "/": error sending request for url (http://127.0.0.1:8000/): error trying to connect: tcp connect error: Only one usage of each socket address (protocol/network address/port) is normally permitted. (os error 10048)

```

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [June 25, 2021, 10:00pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/19 "2021-06-25T22:00:02Z")

</div>

Please make a PR.  
Mac was giving different errors, but similar.

---

<div class="post-metadata">

**Author:** ![essenciary](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/essenciary/32/210469_2.png) [@essenciary](https://discourse.julialang.org/u/essenciary)\
**Post date:** [June 25, 2021, 10:05pm UTC](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541/20 "2021-06-25T22:05:54Z")

</div>

Using Genie’s own HTML and JSON renderers it actually beats HTTP.jl times on my machine, so it looks like that’s the best way to run Genie

Genie prod with own renderers

 ![image](https://global.discourse-cdn.com/julialang/original/3X/b/2/b22794dc0ad9e9614c97208221ae3418726eeef7.png)

[Next page](https://discourse.julialang.org/t/goose-attack-against-a-genie-app/63541.md?page=2)