Is there a way to make a julia variable locally global (for the lack of a better word) ?
To give a little context, I am currently trying to implement some inter-process-communication using julia.
With what I know (understand very little) about this process, I need to read a named pipe multiple times from a given server.
My example code here is shown again below
using MsgPack, Parameters, Sockets
struct TestStructure
a::Int64
b::Vector{Float64}
end
# Prepare MsgPack to understand "TestStructure" structures
MsgPack.msgpack_type(::Type{TestStructure}) = MsgPack.StructType()
function MsgPack_server()
# Initialize named pipe server
server = listen("\\\\.\\pipe\\JuliaPipe")
# Keep waiting for instructions
while true
# Connect to server
sock = accept(server)
# Wait until reception job is finished
while isopen(sock)
# Read data
msg = readline(sock)
# De-code data
test = unpack(msg, TestStructure) # <--- in the end how to replace this using information coming through the pipe
# Print data
println(test)
# Close connection
close(sock)
end
# Start new connection to server
sock = accept(server)
# Wait until send job is finished
while isopen(sock)
# Perform some computation
test2 = TestStructure(3,rand(Float64,5))
# Code data
msg = pack(test2)
# Write data to stream server
write(sock, msg)
flush(sock)
# Close connection
close(sock)
# Print data
println(test2)
end
return nothing
end
The variable test is nested in two while-loops so it is not accessible from a higher scope.
However, for my use case, I would like to access such streamed variable, so I can actually use them in some computations. So, how should I make such variables accessible from another iteration of the second while-loop hence the locally global.
As far as I understand this (please correct me if I am wrong), a let block protects a local declaration from a global one, and probably in the same manner protects the global variable from the local declaration.
However, I can’t figure how it would help trespass such scope protection…
I am doing IPC in the following manner : connect to pipe -> read stream -> do something with the streamed data. Sometimes in send
strings : which are used to determine what computation should be launched (within the MsgPack_server function)
custom structures : which are used as arguments for the computations
Each time I send an information through the pipe, one iteration of the while isopen(sock) loop runs, making the passed data unavailable for the next iteration e.g. :
First iteration : a specific struct is sent over the pipe. test is of type mystruct1 using the type I can store test with appropriate if ...elseif ... end blocks such that ms1 = test
Second iteration : a string is sent over the pipe, let’s say test is of type String and test = "DoSomethingWithMs1" . Again, with appropriate conditions, I can start a ms2 = DoDomethingWithMs1(ms1). But for that I have to know ms1
So basically, I need persistence of previous-iteration variables inside my MsgPack_server() call.
I am sorry, I don’t see what you mean. Currently I am doing something like
while isopen(sock)
msg = readline(sock)
test = TypeAwareUnpack(msg)
if typeof(test) == String
if test == "DoSomethingWithMs1"
ms2 = DoSomethingWithMs1(ms1)
end
else
if typeof(test) == Mystruct1
ms1 = test
end
end
Is this what you are thinking about (at least that’s how I understand your last comment)
I think I am starting to see where you are going with this, but in the end I am not sure if it will make much of a difference than using the local proposed by @lmiq.
Wrapping in a function would probably improve performance a bit though.
Anyway, thanks for the advice.
The decomposition that @Tamas_Papp is recommending will not only possibly improve performance, but will definitely increase reusability, composability, and maintainability by following the principle of separation of concerns. That is, separate the IPC logic from the application (ie., compute) logic.
As I said before, this is surely a naive implementation, and I am all ears for advice. I just do not see at the moment what this decomposition would look like, that’s all. If you have an example for a good practice, feel free to post it .
One approach would be split your function into two, each containing one of the inner while loops. Call one readPipe and the other writePipe. Then call those like this:
data = readPipe()
results = computeSomething(data)
writePipe(results)
Then your outer while loop could wrap those, or better, put those three lines in a function and calling that function could be the body of your outer while loop.
Another approach could be to use three separate Tasks, two for reading and writing to the pipe and another for computing the results. The Tasks would communicate via Channels. Asynchronous Programming · The Julia Language
function readPipe(server)
sock = accept(server)
while isopen(sock)
msg = readline(sock)
data = unpack(msg)
close(sock)
end
return data
end
function writePipe(server,data)
sock = accept(server)
while isopen(sock)
msg = pack(sock, data)
close(sock)
end
end
function read_compute_write(server)
data = readPipe(server)
results = computeSomething(data)
writePipe(server,results)
end
function MsgPack_server()
server = listen("path_to_pipe")
while true
read_compute_write(server)
end
end
After writing my last post, I think I understand now how my first one was ambiguous.
I actually need to reuse variables output by the readPipe in numerous iterations of the overall while and not necessarily in the subsequent writePipe e.g
First iteration : data in pipe is of type Data1
data1 = readPipe()
Second iteration : data in pipe is of type Data2
data2 = readPipe()
Third iteration : data in pipe is of type String
result1 = computeSomething(data1,string)
Fourth iteration : data in pipe is of type String
result2 = computeSomething(data1,result1,string)
So it feels like the previous code factorization is not applicable in this case (or more probably this factorization is a bit too aggressive) while the local works fine. Any ideas to improve this ?
Your code between readPipe and writePipe should provide all the necessary logic for determining what actions to take based on the type(s). readPipe and writePipe are still decomposed properly. Don’t intertwine the app logic with the IO logic.
Options for how to accumulate the data read from readPipe would be to put the data in some combination of mutable structs, Dicts, (named) tuples, arrays, or others, depending on your problem. You can either pass the entire collection to computeSomething and have it deal with the multiple options, or have multiple computeSomething methods and deal with the options before calling it.
In short, break the problem into smaller distinct parts and solve those separately. You’ll discover that your problems and solutions will be reusable. Resist temptation to throw it all in a big pile because the pile will become unwieldy fast.
Messaging should only take / return one argument (if server is omitted) : the byte package / the data passed
Filtering conditions based on data may be performed inside or outside of computeSomething as convenience.
Variables that should be accessible in the main loop should be so through an appropriate container. For instance
mutable struct MyOverallContainer
data1::Data1
data2::Data2
result1::Result1
end
Which would then be passed and modified by each computeSomething call.
By the way I think this would not work for me because all of my structures are actually immutables so I wouldn’t know how to initialize this starting MyOverallContainer, but a Dict would surely work.
Do you agree with this summary ?
Yeah, that was my plan, but I just came up with this code like few hours of work ago, so understanding was the priority.
