[pre-ANN] RestClient.jl

The usual guideline for pluralized names (“Packages that provide most of their functionality in association with a new type”) doesn’t apply here, and I don’t think “RestClients” adds any clarity regarding the functionality of the package - if anything, it could mislead users into thinking it’s an instance of this guideline (that it works by creating a RestClient type).

I slightly prefer RestClientScaffold, but I can see the argument for it being too long. Any name is ultimately going to involve some tradeoffs and compromises, and RestClient.jl seems a perfectly fine choice.

Can this package handle with binary format data in communication that support WebSocket, TCP protocol ? The data in the body of the packet is Protobuf codec.

This. Pluralizing the package name in this way would be a mistake.

Here’s another formulation of the same general naming advice:

:check_mark: CONSIDER using plural namespace names where appropriate.

I think a plural S is almost always applicable - we are stuck with the name forever - the cost of an extra S is negligible. The cost of singular is potentially added friction down the line.

It’s not appropriate here! I don’t even know why we’re having this discussion!

Because we disagree :blush:

Just to clarify, and sorry for repeating an argument made in the General registry PR, but I guess my main argument for the pluralised version is that I do think that, at least eventually, there will be a need for a type of which you can create multiple instances (multiple clients), cf., e.g., connecting to multiple services with the same API.

It would also be nice with a poll that includes the other suggested names, e.g., RestClientScaffold, RestClientMacros (and multiple choice, please). Not sure if there has been other suggestions?


Edit:

I got some input from long-time community members supporting RestClientScaffold, or a non-systematic name - rather than RestClient/RestClients - similar to @digital_carver.

So I’ll dip a toe into an alternative poll:

  • Some non-systematic name
  • RestClientScaffold
  • RestClientMacros
  • RestClient
  • RestClients
0 voters

Update: RestClient.jl has just had its registration go through, with a few improvements to the docs/API thanks to some feedback from @stemann :slight_smile:

I hope it will prove useful for anybody looking to wrap a REST-ish API in Julia.

In the future, I’d love to add a framework to make common authentication schemes easier, and with the upcoming Pkg.jl app support I think it would be really neat to make a code-generator tool that produced a package for a particular API from an OpenAPI definition.

I’d like to get to these eventually, but in the meantime PRs are welcome!

If you’ve tried RestClient.jl and found that it wasn’t handling rate-limiting properly for an API you tried, or that the debugging help was less useful that you expected, or that you didn’t see the expected kind of error for a failed request, I’d encourage you to check out v1.0.2 that’s just been released :slight_smile:

I’ve mentioned the debugging a few times I think, but never actually shared a screenshot, so as an incentive/taster, here’s what you’ve been missing out on:

RestClient v1.1 is just out, and don’t let the minor version bump fool you, it’s a pretty substantial release.

Features

F.1 Streaming responses

I’ve had a think about how to make it as easy as possible to support streamed responses, through methods like SSE and NDJSON (which OOTB support is provided for). I’ve landed on a convenient wrapper type: Stream{T} for a stream of T objects that uses a Channel internally and supports iteration.

This means all it takes to read responses from a streaming API is:

@jsondef struct Delta
    type::String
    text."delta.text"::Union{String, Nothing} = nothing
end

# take a prompt string and post it to /messages, to get a stream of Deltas
@endpoint chat(prompt::String) -> prompt -> :post("messages") -> Stream{Delta}

for msg in chat("send me messages")
     println(msg.text)
end

If you want the full SSE frame data, you can just ask for a Stream{SSEvent{T}} and you’ll get SSEvent{T}s out of your iterator.

If you want to close a stream early, you can call close on the stream in another task.

F.2 HTTP form and multipart payloads

In addition to @jsondef and @xmldef, you can now define form and multipart payloads with @formdef and @multipartdef:

@formdef struct LoginInfo
    username::String
    password::String
    rememberme."remember-me"::Bool = false
end

@jsondef struct UserInfo ... end

@endpoint login(auth::LoginInfo) -> auth -> :post("login") -> UserInfo

F.3 Authentication convenience

APIs that take authentication details often use basic, bearer, header, or query based authentication. Previously, RestClient required you to provide this in every endpoint manually, but no longer.

@globalconfig RequestConfig(...) auth=BearerAuth() # or,
@globalconfig RequestConfig(...) auth=(QueryAuth("api_key"), :required)

If you specify the authentication in @globalconfig, all @endpoints defined will use it by default (as always, this can be customised per-endpoint). To make public/authenticated API splits easy though, @endpoint now supports a :auth/:noauth/:authoption keyword:

@endpoint :noauth userinfo(...) -> ...
@endpoint :auth login(...) -> ...

Improvements

  • Support for JSON.jl, while keeping the existing support for JSON3.jl
  • More structured errors, with MalformedRequest for requests that fail validation and ResponseErrors for unsuccessful calls
  • Better rate limiting with an atomic deadline
  • More thorough validation, as validate may now return a whole list of issues
  • RequestConfig will try to avoid printing its secret key (if set) in the REPL, while still supporting repr round-tripping
julia> RequestConfig("https://api.example.com", key = "keepmesecret")
RequestConfig("https://api.example.com"; key = "*****", cache = true)

Fixes

  • getxattr error detection on Linux
  • setexpiry erroring on non-unix systems
  • handling of function keyword arguments in @endpoint

Other changes

  • We now take a bytes-first approach to data type interpretation over IO (the previous approach), which made streaming support a lot easier