# HTTP Streaming Antropic Claude AI

**URL:** <https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666>\
**Category:** General Usage\
**Tags:** http, streaming\
**Created:** [July 31, 2024, 8:23am UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666 "2024-07-31T08:23:30Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [July 31, 2024, 8:23am UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/1 "2024-07-31T08:23:30Z")

</div>

Hey Julianners,  
Yesterday I stucked with the HTTP streaming functionality very well and just cannot figure out how to solve it. I suppose the problem is with the body in the POST message. Also I cannot do `write(response, body)` due to it also gives back error in the end. Also tried the `response_stream`

```julia
io=IOBuffer();
HTTP.post(url, headers, JSON.json(body); response_stream=io)

```

but in this case I also get all the response at once instead of one by one.

The example code is here:

```julia
using HTTP
using JSON

url = "https://api.anthropic.com/v1/messages"

body = Dict(
    "messages" => [
        Dict("content" => "Hi, tell me a very short story", "role" => "user"),
    ],
    "model" => "claude-3-5-sonnet-20240620",
    "max_tokens" => 256,
    "stream" => true, # Set to true for streaming
)
headers = Dict(
    "content-type" => "application/json",
    "x-api-key" => ENV["ANTHROPIC_API_KEY"],
    "anthropic-version" => "2023-06-01",
)
function stream_response(url, headers, body)
    HTTP.open("POST", url, headers, body=body) do response
        @show response
        for line in eachline(response)
            @show line
            if startswith(line, "data: ")
                data = JSON.parse(line[6:end])
                if data["type"] == "content_block_delta" && data["delta"]["type"] == "text_delta"
                    println(data["delta"]["text"])
                    flush(stdout)
                end
            end
        end
    end
end
stream_response(url, headers, JSON.json(body))

```

It must be working as a curl command does just right:

```julia
curl https://api.anthropic.com/v1/messages \
     --header "anthropic-version: 2023-06-01" \
     --header "content-type: application/json" \
     --header "x-api-key: $ANTHROPIC_API_KEY" \
     --data \
'{
  "model": "claude-3-5-sonnet-20240620",
  "messages": [{"role": "user", "content": "Hello, tell me a very short story"}],
  "max_tokens": 256,
  "stream": true
}'

```

Any idea how to manage it then?

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [August 5, 2024, 2:28pm UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/2 "2024-08-05T14:28:28Z")

</div>

Generated a workaround with AI. If someone would bump into this issue. (Also note that the data comes in larger batches this way than the curl version when it is ran naturally in the terminal.)

```julia
stream_anthropic_response(prompt::String; model::String="claude-3-5-sonnet-20240620", max_tokens::Int=256) = stream_anthropic_response([Dict("role" => "user", "content" => prompt)]; model, max_tokens)
function stream_anthropic_response(msgs::Vector{Dict{String,String}}; model::String="claude-3-5-sonnet-20240620", max_tokens::Int=256)
    body = JSON.json(Dict(
        "model" => model,
        "messages" => msgs,
        "max_tokens" => max_tokens,
        "stream" => true
    ))
    `curl -sS -N https://api.anthropic.com/v1/messages -X POST -H anthropic-version: 2023-06-01 -H content-type: application/json -H x-api-key: $(ENV["ANTHROPIC_API_KEY"]) -d $body`
    
    for line in eachline(pipeline(cmd))
        startswith(line, "data: ") || continue
        data = JSON.parse(replace(line, r"^data: " => ""))
        data == "[DONE]" && break
        
        if get(get(data, "delta", Dict()), "type", "") == "text_delta"
            text = data["delta"]["text"]
            print(text)
            flush(stdout)
        end
    end
end

```

I guess the larger batch are some kind of catching at some layer. Would be happy if someone would have idea about it. But of course the Opus 3.5 maybe will solve/answer this anyways ofc… xD

I also send the above issue to the [HTTP.jl - github](https://github.com/JuliaWeb/HTTP.jl) the opened: [Issue](https://github.com/JuliaWeb/HTTP.jl/issues/1185)  
The HTTP.jl should work naturally I believe.

---

<div class="post-metadata">

**Author:** ![svilupp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/svilupp/32/34933_2.png) [@svilupp](https://discourse.julialang.org/u/svilupp)\
**Post date:** [August 7, 2024, 7:45am UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/3 "2024-08-07T07:45:44Z")

</div>

I don’t think you can do it by newlines.

Have you looked at how OpenAI.jl implements it? It should be exactly the same: [OpenAI.jl/src/OpenAI.jl at 762daadd3ad35ae1badd47fdc02796ca7bd6c886 · JuliaML/OpenAI.jl · GitHub](https://github.com/JuliaML/OpenAI.jl/blob/762daadd3ad35ae1badd47fdc02796ca7bd6c886/src/OpenAI.jl#L91)

In short, you

- open a streamed POST request in a do-block.
- write your body.
- close writer
- open reader
- iterate with while using `eof` or DONE signal
- close reader

It’s same as here in HTTP docs: [Client · HTTP.jl](https://juliaweb.github.io/HTTP.jl/stable/client/#Streaming-Requests)

Difference I see in your first implementation

- passing body directly - I think it has to be passed into the stream
- reading the stream safely `readavailable` (it doesn’t always come separated by new lines)
- wait for DONE (might be OpenAI thing?)

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [August 8, 2024, 7:06pm UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/4 "2024-08-08T19:06:43Z")

</div>

I was getting Error like:

```julia
ERROR: HTTP.RequestError:
HTTP.Request:
HTTP.Messages.Request:
"""
POST /v1/messages HTTP/1.1
anthropic-version: 2023-06-01
content-type: application/json
x-api-key: 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
Host: api.anthropic.com
Accept: */*
User-Agent: HTTP.jl/1.10.4
Accept-Encoding: gzip
Transfer-Encoding: chunked

[Message Body was streamed]"""Underlying error:
IOError: read: connection reset by peer (ECONNRESET)
...

```

But indeed in the OpenAI.jl they used the:  
`HTTP.closewrite(io)` and `HTTP.startread(io)`

So with your guide the working code:

```julia
using HTTP
using JSON

body = Dict(
    "messages" => [
        Dict("content" => "Hi, tell me a longer story", "role" => "user"),
    ],
    "model" => "claude-3-5-sonnet-20240620",
    "max_tokens" => 256,
    "stream" => true, # Set to true for streaming
)
headers = Dict(
    "content-type" => "application/json",
    "x-api-key" => ENV["ANTHROPIC_API_KEY"],
    "anthropic-version" => "2023-06-01",
)
HTTP.open("POST", "https://api.anthropic.com/v1/messages", headers; status_exception=false) do io
	write(io, JSON.json(body))
	HTTP.closewrite(io) # indicate we're done writing to the request
	HTTP.startread(io) 
	while !eof(io)
		println(String(readavailable(io)))
	end
	HTTP.closeread(io)
end

```

Really THANK YOU for the help! Beautiful to see that this is also working perfectly in julia!

(Note: Sonnet 3.5 is overloaded like hell… so it is really hard to test and see if it works. But it works I know because It worked once when their server was up for a second. )

---

<div class="post-metadata">

**Author:** ![svilupp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/svilupp/32/34933_2.png) [@svilupp](https://discourse.julialang.org/u/svilupp)\
**Post date:** [August 8, 2024, 7:16pm UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/5 "2024-08-08T19:16:22Z")

</div>

Do I see the Anthropic.jl forming? 🥸

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [August 9, 2024, 8:37am UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/6 "2024-08-09T08:37:24Z")

</div>

Defnitely! 😃 I will make a really basic Anthropic.jl fast yes. 😃

OR I mean the Anthropic Sonnet 3.5 will create it itself for “itself” XD

---

<div class="post-metadata">

**Author:** ![Marcell\_Havlik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marcell_havlik/32/37424_2.png) [@Marcell\_Havlik](https://discourse.julialang.org/u/Marcell_Havlik)\
**Post date:** [August 9, 2024, 10:38am UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/7 "2024-08-09T10:38:00Z")

</div>

> **[GitHub - Cvikli/Anthropic.jl: Little extension for the...](https://github.com/Cvikli/Anthropic.jl)**
>
> Little extension for the https://github.com/svilupp/PromptingTools.jl to have streaming

On the way. Hope it can be more helpful later on 🙂

---

<div class="post-metadata">

**Author:** ![ametalci](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ametalci/32/211476_2.png) [@ametalci](https://discourse.julialang.org/u/ametalci)\
**Post date:** [August 22, 2024, 5:38am UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/8 "2024-08-22T05:38:55Z")

</div>

Thank you for this working example. I’ve been searching for days just to find how to get streaming output from Ollama in Julia 😀

NOTE: Oh I’ve been using PromptingTools.jl of yours @svilupp

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [May 29, 2026, 10:14pm UTC](https://discourse.julialang.org/t/http-streaming-antropic-claude-ai/117666/9 "2026-05-29T22:14:01Z")

</div>

I missed this post originally (sorry!), but just wanted to drive by comment that HTTP.jl now supports (as of like 6 months or so) an `sse_callback` `HTTP.request` keyword arg where you can pass a callback that is invoked on each `HTTP.SSEEvent` that is parsed/read from a text/event-stream response body. The sse events are parsed automatically and all you have to supply is the callback to decide what to do on each event. I use it myself in anthropic/openai/llm [response streaming](https://github.com/quinnj/Agentif.jl/blob/57977e7f4be999dca404ce283ccd85b81e70f302/Agentif/src/stream.jl#L378) code.