How to correctly stream data over a pipe to account for type variability?

New to Julia
1

I (very) recently managed to stream data from/to another program with julia through a named pipe.
The base example is below but basically, what I can do now is :

  • Declare some custom types to stream, on both sides
  • Read and write objects associated with these types
  • Pack/Unpack correctly these objects.

However, as I do not always pack/unpack (using MsgPack) the same objects, I need to have a way to tell julia (and of course the other program, but that’s another story) what the incoming types are at runtime.

How can I send first the type of the data to be decoded, then the data itself ?

using MsgPack, Parameters, Sockets

struct TestStructure
    a::Int64
    b::Vector{Float64}
end

# Prepare MsgPack to understand "TestStructure" structures
MsgPack.msgpack_type(::Type{TestStructure}) = MsgPack.StructType()


function MsgPack_server()

    # Initialize named pipe server
    server = listen("\\\\.\\pipe\\JuliaPipe")
    # Keep waiting for instructions
    while true
        # Connect to server
        sock = accept(server)
        # Wait until reception job is finished
        while isopen(sock)
            # Read data
            msg = readline(sock)
            # De-code data
            test = unpack(msg, TestStructure)  # <--- in the end how to replace this using information coming through the pipe
            # Print data
            println(test)
            # Close connection
            close(sock)
        end
        # Start new connection to server
        sock = accept(server)
        # Wait until send job is finished
        while isopen(sock)
            # Perform some computation
            test2 = TestStructure(3,rand(Float64,5))
            # Code data
            msg = pack(test2)
            # Write data to stream server
            write(sock, msg)
            flush(sock)
            # Close connection
            close(sock)
            # Print data
            println(test2)
        end
    return nothing
end