Performance issue with an API calling another API

Specific Domains Web Stack
, , , ,
1

Hi everyone,

Exploring the Oxygen package to build APIs I found this I haven’t been able to solve

  1. When I run this API (with a 150 ms sleep to simulate the time taken by a model):
using Oxygen

@get "/model" function()
    sleep(0.15)
    return json("ok")
end

serve(port = 8001)

I reach about 560 RPS (with concurrency = 100)

ab -n1000 -c100 'http://localhost:8001/model'

Server Software:        
Server Hostname:        localhost
Server Port:            8001

Document Path:          /model
Document Length:        4 bytes

Concurrency Level:      100
Time taken for tests:   1.756 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      83000 bytes
HTML transferred:       4000 bytes
Requests per second:    569.62 [#/sec] (mean)
Time per request:       175.556 [ms] (mean)
Time per request:       1.756 [ms] (mean, across all concurrent requests)
Transfer rate:          46.17 [Kbytes/sec] received

When I build another Oxygen API consuming the previous one

using Oxygen
using HTTP

@get "/test" function()
    response = HTTP.get("http://localhost:8001/model")
    return json("ok")
end

serve(port = 8002)

I reach only around 100 RPS even using 4 threads with serveparallel. I would expect to see an RPS number closer to 560.

ab -n1000 -c100 'http://localhost:8002/test'

Server Software:        
Server Hostname:        localhost
Server Port:            8002

Document Path:          /test
Document Length:        4 bytes

Concurrency Level:      100
Time taken for tests:   9.815 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      83000 bytes
HTML transferred:       4000 bytes
Requests per second:    101.89 [#/sec] (mean)
Time per request:       981.455 [ms] (mean)
Time per request:       9.815 [ms] (mean, across all concurrent requests)
Transfer rate:          8.26 [Kbytes/sec] received

If I do the same with FastAPI

import requests
from fastapi import FastAPI

app = FastAPI()

@app.get("/test")
def test():
    response = requests.get("http://localhost:8001/model")
    return "ok"

With just one worker (uvicorn api_test_nested:app --host 0.0.0.0 --port 8003 --workers 1) I get around 220 RPS

ab -n1000 -c100 'http://localhost:8003/test'

Server Software:        uvicorn
Server Hostname:        localhost
Server Port:            8003

Document Path:          /test
Document Length:        6 bytes

Concurrency Level:      100
Time taken for tests:   4.436 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      149000 bytes
HTML transferred:       6000 bytes
Requests per second:    225.42 [#/sec] (mean)
Time per request:       443.618 [ms] (mean)
Time per request:       4.436 [ms] (mean, across all concurrent requests)
Transfer rate:          32.80 [Kbytes/sec] received

I think the issue is in the HTTP.get request on the Julia API maybe there is a better way to do it or ways to speed it up. Any hint is welcome.

Cheers,

Abraham

2

Progress:

I changed the /test API to use curl instead of HTTP.jl and I got a huge improvement so I’m almost sure there is something strange with HTTP.jl

using Oxygen

@get "/test" function()
    response = run(`curl http://localhost:8001/model`) # instead of HTTP.request("GET", "http://localhost:8001/model")
    return json("ok")
end

serve(port = 8002)

With this small change I go from 100 RPS to about 440 RPS, much closer to 560 RPS I get from /model and twice the RPS from FastAPI with one worker.

Any suggestion is welcome!