# How to handle HTTP.Exceptions.StatusError

**URL:** <https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649>\
**Category:** Web Stack\
**Tags:** http\
**Created:** [July 15, 2023, 10:42am UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649 "2023-07-15T10:42:03Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 15, 2023, 10:42am UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/1 "2023-07-15T10:42:03Z")

</div>

I am getting HTTP.Exceptions.StatusError

```julia
using CSV, DataFrames,HTTP

```

![image](https://global.discourse-cdn.com/julialang/original/3X/9/c/9c7568378ac9d3ecbe9cf4a3a0b48df422e52026.png)

```julia
begin
	for x in 33047:33187
            url="https://gcn.nasa.gov/circulars/$x"
            txt=String((HTTP.get(url)))
        if occursin("report on behalf of the Swift/UVOT team",txt)
             hb,he=findfirst(r"^Filter"im,txt)
			 lr,_=findnext("\n\nThe",txt,he)
			 cltxt=replace(txt[hb:lr], " +/- "=>"+/-", r" +(\w)"=>s"\t\1",r" +(&gt;)"=>s"\t>")
			 df=CSV.read(IOBuffer(cltxt), DataFrame, delim='\t')
			 CSV.write("data.csv" ,df)
        end
    end
end

```

I tried with `try` and `catch` block but it was not working.

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [July 15, 2023, 11:20am UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/2 "2023-07-15T11:20:08Z")

</div>

Try in this way:

```julia
using HTTP
begin
    for x in 33047:33187
        try
            url = "https://gcn.nasa.gov/circulars/$x"
            resp = HTTP.get(url)           
            @info "processing $url with status $(resp.status)"
            txt = String(resp)
            if occursin("report on behalf of the Swift/UVOT team", txt)
                hb, he = findfirst(r"^Filter"im, txt)
                lr, _ = findnext("\n\nThe", txt, he)
                cltxt = replace(txt[hb:lr], " +/- " => "+/-", r" +(\w)" => s"\t\1", r" +(&gt;)" => s"\t>")
                df = CSV.read(IOBuffer(cltxt), DataFrame, delim='\t')
                CSV.write("data.csv", df)
            end
        catch e
            @error "error at $x" e
        end
    end
end

```

I only show the error in the `catch` branch, not rethrow anything.

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 15, 2023, 11:57am UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/3 "2023-07-15T11:57:35Z")

</div>

```julia
CSV.write("data.csv", df)

```

this saves only last _x_ output in CSV but i need all with _x_ value in extra column.

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [July 15, 2023, 12:10pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/4 "2023-07-15T12:10:14Z")

</div>

I didn’t alter your code in any way - I just handled the error.

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 15, 2023, 12:21pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/5 "2023-07-15T12:21:04Z")

</div>

CSV overwrites by default. Use  
`CSV.write("data.csv",df;append=true)`  
to concatenate all x outputs.

You may also have to add `writeheader=true` on the first output (it default to `false` when `append` is `true`)

Hope that helps,

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 15, 2023, 12:24pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/6 "2023-07-15T12:24:35Z")

</div>

@jdad Yes , How to add extra column for storing GCN circular _x_ value ?

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 15, 2023, 4:06pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/7 "2023-07-15T16:06:42Z")

</div>

Try adding the statement `df.x=x` just before the CSV.write?

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 15, 2023, 4:12pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/8 "2023-07-15T16:12:20Z")

</div>

Is that `writeheader=true` correct ? code is still running after 15 minutes. 😫

```julia
begin
    for x in 33037:33187
        try
            url = "https://gcn.nasa.gov/circulars/$x"
            resp = HTTP.get(url)            
            txt = String(resp)
            if occursin("report on behalf of the Swift/UVOT team", txt)
                hb, he = findfirst(r"^Filter"im, txt)
                lr, _ = findnext("\n\nThe", txt, he)
                cltxt = replace(txt[hb:lr], " +/- " => "+/-", r" +(\w)" => s"\t\1", r" +(&gt;)" => s"\t>")
                df = CSV.read(IOBuffer(cltxt), DataFrame, delim='\t')
				df.x=x
                CSV.write("data.csv", df; append=true; writeheader=true)
            end
        catch e
            @error "error at $x" e
        end
    end
end

```

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 15, 2023, 4:17pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/9 "2023-07-15T16:17:58Z")

</div>

This will write header for every iteration. You could try to replace the one-line-statement CSV.write by a 3-line one

```julia
if x==33037
    CSV.write("data.csv", df)
else
    CSV.write("data.csv", df; append=true, writeheader=false)
end
```

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 15, 2023, 4:30pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/10 "2023-07-15T16:30:26Z")

</div>

Note, this suppose that header is the same for every x value. You should check it?

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 15, 2023, 4:31pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/11 "2023-07-15T16:31:38Z")

</div>

Header isn’t same for every table. 🙃 Header content is almost same.

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 15, 2023, 5:16pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/12 "2023-07-15T17:16:22Z")

</div>

What is the difference between header and header content?

Anyway you could write a file for each value of x

`CSV.write("data-$(x).csv",df)`

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 15, 2023, 5:30pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/13 "2023-07-15T17:30:14Z")

</div>

I got no output. Please see code below:

```julia
begin
    for x in 33037:33187
        try
            url = "https://gcn.nasa.gov/circulars/$x"
            resp = HTTP.get(url)            
            txt = String(resp)
            if occursin("report on behalf of the Swift/UVOT team", txt)
                hb, he = findfirst(r"^Filter"im, txt)
                lr, _ = findnext("\n\nThe", txt, he)
                cltxt = replace(txt[hb:lr], " +/- " => "+/-", r" +(\w)" => s"\t\1", r" +(&gt;)" => s"\t>")
                df = CSV.read(IOBuffer(cltxt), DataFrame, delim='\t')
				df.x=x
                CSV.write("data-$(x).csv",df)
            end
        catch e
            @error "error at $x" e
        end
    end
end

```

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [July 15, 2023, 7:17pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/14 "2023-07-15T19:17:43Z")

</div>

I would suggest using `String(resp.body)` instead of `String(resp)`. In that way you’ll only operate on the extracted HTML string.

Please check if your `occursin("report on behalf of the Swift/UVOT team", txt)` condition is true at least once: I just added a print statement inside that `if` block and and I cannot see a single instance where that condition is true.

So, it is expected to not have any file output - because that code never gets executed.

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 16, 2023, 3:08am UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/15 "2023-07-16T03:08:02Z")

</div>

> [@algunion](#):
>
> Please check if your `occursin("report on behalf of the Swift/UVOT team", txt)` condition is true at least once

For _x_ =[33211](https://gcn.nasa.gov/circulars/33211) . It satisfies `occursin` condition but it gives error:

```julia
ArgumentError
msg

"It is only allowed to pass a vector" ⋯ 60 bytes ⋯ "v` if you want to use broadcasting."

```

@jdad i can overcome this ArgumentError by removing `df.x=x` from code but `CSV.write("data-$x.csv",df)` is not saving any file .

Try this code given below:

```julia
begin
    for x in 33110:33511
        try
            url = "https://gcn.nasa.gov/circulars/$x"
            resp = HTTP.get(url)            
            txt = String(resp.body)
            if occursin("report on behalf of the Swift/UVOT team", txt)
                hb, he = findfirst(r"^Filter"im, txt)
                lr, _ = findnext("\n\nThe", txt, he)
                cltxt = replace(txt[hb:lr], " +/- " => "+/-", r" +(\w)" => s"\t\1", r" +(&gt;)" => s"\t>")
                df = CSV.read(IOBuffer(cltxt), DataFrame, delim='\t')	
                @show df
				CSV.write("data-$x.csv",df)
            end
        catch e
            @error "error at $x" e
        end
    end
end

```

---

<div class="post-metadata">

**Author:** ![jdad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdad/32/4739_2.png) [@jdad](https://discourse.julialang.org/u/jdad)\
**Post date:** [July 16, 2023, 8:22am UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/16 "2023-07-16T08:22:04Z")

</div>

This code works for me

```julia
using HTTP,DataFrames,CSV

function doanalysis()
dfg=nothing
for x in 33037:33287 # was 33037:33187, but mention of 33211 as good candidate
    print("\r peeking at $x ")
    try
        url = "https://gcn.nasa.gov/circulars/$x"
        resp = HTTP.get(url) 
        status=resp.status
        print(" ",status," "); # println();
        if status == 404 ; println("status=",status); continue; end          
        txt = String(resp.body)
        if occursin("report on behalf of the Swift/UVOT team", txt)
            println(" Swift report")
            # println()
            # @info "this is a Swift report"
            hb, he = findfirst(r"^Filter"im, txt)
            lr, _ = findnext("\n\nThe", txt, he)
            cltxt = replace(txt[hb:lr], " +/- " => "+/-", r" +(\w)" => s"\t\1", r" +(>)" => s"\t>")
            cltxt = replace(cltxt,"&gt;" => "\t>")
            # println("cltxt=");print(cltxt)
            df = CSV.read(IOBuffer(cltxt), DataFrame, delim='\t')
            df.x=[x for i in 1:nrow(df)]
            # println();describe(df); show(df);println()
            if isnothing(dfg) # x == 33037
                dfg=df
            else
                dfg=vcat(dfg,df)
            end # if x is first
            CSV.write("data-$(x).csv",df)
        end # if occursin
    catch e
        println("error ")
        # @error "error at $x" # e
    end # trycatch
end # for loop
println()
if !isnothing(dfg)
    CSV.write("data-all.csv",dfg)
else
    @info "no dfg to write"
end # !isnothing
end # function doanalysis
doanalysis()
# @info "tkumar done"
println("tkumar done.")

```

---

<div class="post-metadata">

**Author:** ![raman\_kumar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raman_kumar/32/26782_2.png) [@raman\_kumar](https://discourse.julialang.org/u/raman_kumar)\
**Post date:** [July 16, 2023, 5:17pm UTC](https://discourse.julialang.org/t/how-to-handle-http-exceptions-statuserror/101649/17 "2023-07-16T17:17:10Z")

</div>

- For `if occursin("report on behalf of the Swift/UVOT team", txt)` then also add a column titled _Telescope_ containing **SWIFT/UVOT** in each row.
- For `if occursin("MASTER-OAFA robotic telescope ", txt)` then also add a column titled _Telescope_ containing **Global MASTER-Net** in each row.