Using the Sockets Library

General Usage
1

Hello, i have been having trouble with the sockets library.
For some reason when i write to a connection it doesnt succeed and instead hangs. I have no idea why this is happening, would anyone else have a clue?

function start(port)
        server = Sockets.listen(port)
        while true
          conn = Sockets.accept(server)
          @async begin
            println("Connection Start")
            processConnection(conn)
          end
        end
    end
    function processConnection(conn)
        data = Vector{UInt8}()
        try
            while true
                tmp = read(conn)
                if(length(tmp) > 0)
                    append!(data,tmp)
                else
                    break
                end
                println("here")
            end
        catch err
            print("connection ended with error $err")
        end
        println(data)
        if(length(data) == 0)
            return 
        end
        println(ntoh(data[1]), " ",data[1])
        transit_data = Transit.decode(data)
        buffer = processTransitData(transit_data)
        println("ABOUT TO WRITE")
        write(conn,buffer)
        println(conn," ",buffer)
        close(conn)
    end
2

After some more testing it appears to be that the “write” to the socket doesnt actually write until the program ends on its own. I have no clue why this may be.

3

read blocks. The example from Sockets Library issue: write delays until program ends · Issue #40705 · JuliaLang/julia · GitHub works as you expect if you use something like this instead:

Server:

using Sockets
server = Sockets.listen(8080)
conn = Sockets.accept(server)
while !eof(conn)
    data = readavailable(conn) # <-- !!!
    println(data)
    write(conn, data)
end

Client:

using Sockets
client = Sockets.connect(8080)
write(client, "hello")
println(readavailable(client)) # <-- !!!
2 Likes
4

Thank you, i felt like it was something i was doing wrong

5

Actually i seem to be having a different issue now. For some reason, readavailable() is blocking when there is no data available.
Any reason why that may occur?

6

The documentation is a bit ambiguous but that seems to be the intended behavior: read whatever data is available but block until some data is available. Perhaps @jameson can comment on whether that’s what was intended (IIRC, he wrote this code originally).

7

I’ve been using readavailable() for years and it has always blocked. The way I use it is in an async task with a channel for communication.

8

Probably not me: I usually tell people that readavailable is a foot-gun, and not to use. And if you think you need it, you should go back through the design and and check for accidental data handling errors (e.g. races and deadlocks). The error here with OP is forgetting to close the write side after being done with it (that said, we should someday finally implement half-open / shutdown state sockets–but they aren’t that commonly used since they’re often redundant with high-level protocols). Currently I think you have to ccall uv_shutdown manually now, which involves a bit of boilerplate unfortunately.

1 Like
9

Hello, the problem was that i needed to wait for a reply, but the original write never ended up being sent until the socket closed. I thought TCP sockets were meant to just send when written too? Or do they require a “send” after writing to the socket first? Sorry im not the best with TCP sockets

10

You do seem to have a convoluted processConnection function with a lot of redundancy. Why not shorten it to the (functionally equivalent) form below:

    function processConnection(conn)
        data = read(conn)
        println(data)
        if isempty(data)
            return 
        end
        ...

Sockets do require a “send” call, known as “shutdown” or “half-close” if you are not using a higher level protocol. This is the ccall(:uv_shutdown) call that I mentioned above is not currently available.

11

Thanks for the improvement, ill add it in right away.
But what would happen if a large amount of data was sent in multiple packets? Couldnt it read while stuff is still being sent and miss the end of the message?
Also are you saying i should be calling ccall(:uv_shutdown) after my write code? Or is it fine with the higher level library?

12

TCP does not expose the concept of packets. You can either use a higher level protocol (like HTTP) or call uv_shutdown after your write. Unfortunately, though uv_shutdown isn’t quite trivial to wrap.

13

Where would i find information on that uv_shutdown command? im looking for what its missing and cant seem to find it.
Also, if i want to write to the socket again would i need to do some form of “startup” command? or is that unneeded?

14

It doesn’t really exist now, so you would need to use an existing library that also provides packet framing (ZMQ.jl, Protobuf.jl, HTTP.jl, Serialization.jl, etc). I wrote the code for supporting it just now: add stream shutdown and support half-duplex operation by vtjnash · Pull Request #40783 · JuliaLang/julia · GitHub

2 Likes