Comma in a query HTTP.jl

Specific Domains Web Stack
,
1

Spoiler: This is my first time doing any sort of web-interfacing, so might be something obvious.

Problem:

Hard-typing the query into a request works, while using the query=["variables"=>"var1,var2"] or query=["variables"=>"var1", "variables"=>"var2"] causes an error on the server side due to URI encoding of the comma.

Example

URL access

Using a dummy URL for demonstartion purposes, but the actual address is public, so if really necessary, I can reveal it.

using HTTP

happyrequest=HTTP.get("https://info.mywebsite.com/my.cgi?variables=var1,var2&param=10")

sadrequest=HTTP.get("https://info.mywebsite.com/my.cgi", query=["variables"=>"var1,var2", "param"=10)

running the above with verbose=1

#happyrequest
...
...
Debug: HTTP.Messages.Request:
│ GET /my.cgi?variables=var1,var2&param=10 HTTP/1.1
│ Accept: */*
│ User-Agent: HTTP.jl/1.10.3
│ Content-Length: 0
│ Accept-Encoding: gzip
...
│ HTTP/1.1 200 OK
│ Date: Tue, 08 Apr 2025 13:37:41 GMT
│ Server: Apache/2.4.58 (Unix) OpenSSL/3.2.1 PHP/8.1.27
│ Strict-Transport-Security: max-age=63072000; includeSubDomains
│ Transfer-Encoding: chunked
│ Content-Type: text/html;charset=iso-8859-1
│ 
│ [
│ { "var1": Value1, "var2": Value2 }
│ ]


#sadrequest
...
...
Debug: HTTP.Messages.Request:
│ GET /my.cgi?variables=var1%2Cvar2&param=10 HTTP/1.1
│ Accept: */*
│ User-Agent: HTTP.jl/1.10.3
│ Content-Length: 0
│ Accept-Encoding: gzip
...
│ HTTP/1.1 200 OK
│ Date: Tue, 08 Apr 2025 13:21:43 GMT
│ Server: Apache/2.4.58 (Unix) OpenSSL/3.2.1 PHP/8.1.27
│ Strict-Transport-Security: max-age=63072000; includeSubDomains
│ Transfer-Encoding: chunked
│ Content-Type: text/html;charset=iso-8859-1
│ 
│  [{ "error": "someLib: RelationalDataBase: SQL Error" }]

The only difference between the requests that I see is the URI-encoding of the comma that separates var1,var2 in the query string. I have attempted to use both ways of assignment of the second variable to the same key:

  1. Including it into a comma separated string
  2. Using a dict as per documentation:

HTTP.get(url; query=["x1" => "y1", "x1" => "y2"]: allows duplicate key values

In both cases a URI-encoding of the comma takes place, causing indigestion on the server side.
Question: is there, maybe, a way to disable the encoding?
I can work around it by doing individual requests for each variable or by manipulating the string of the request. However my gut feeling tells me there should be a better way, right?

Thank you for your time and any (potential) contributions or suggestions.

Igor

2

As a stop-gap solution I ended up with:

vars=["var1", "var2"]
happyrequest=HTTP.get("https://info.mywebsite.com/my.cgi?variables=$(join(vars,','))&param=10")

But still hoping for something a bit prettier :upside_down_face:

3

I think commas are just messy; they’re “reserved” so the encoded versions aren’t equivalent. See: symbols - Usage of comma in URL: encoded or not encoded - Webmasters Stack Exchange

It seems best practice would be to encode them, but then your server would need to also follow those best practices to be able to handle it. Note that you probably still want to URL-encode non-comma characters.

1 Like
4

Thanks for the link, it somewhat confirmed my suspicion that it will probably have to be messy if I want it to work. As the server is out my influence and is being used by other clients, the chances for any change in the query encoding (or lack thereof) are very slim.
In any case I am happy with a working code that is not so pretty if the other option is a pretty non-working one…