I’m starting a new project and DETERMINED to learn from my past adventures in Julia (FUN).
I am setting up modules and projects BUT can’t figure out how to cater for the situation where two programs communicate with a third via zmq. Can I put this configuration in one project and if so how. When I
$julia ; ] ; pkg > generate zmq_pull
It creates ONE file zmq_pull .jl as expected. But there are two other scripts
zmq_push_1.jl
zmq_push_2.jl
which are run in separate cron processes and communicate with zmq_pull.jl through ZMQ.
Do I have to create 3 separate projects for each of the component scripts? I don’t see how using include would help me.
I suppose I don’t totally understand what you’re trying to do. It may be some confusion around terms like “project”, and “program”.
In Julia, I usually think of a “project” as the layer that could controls dependencies, whereas a “program” is something that’s going to run. It sounds like you have 3 different scripts that have some shared functionality? You could put the shared functionality into a module or package, and then have your scripts use that?
But perhaps a bit more description about what your program(s) are meant to do, and their relationship to each other would be helpful.
Hi there
thanks for the reply.
The connective relationship between the scripts is ZMQ.jl so the PUSH scripts send messages to the PULL script.
There is NO shared functionality. The PUSH scripts send messages to the PULL script that builds out a dataframe based on the messages.
Each is started in it’s own PID using cron. Right now I’m using PUSH/PULL for simplicity but will use a more scalable approach later on.
So if ZMQ.jl is handling all the functionality, I think having a single project (just to make sure the version of ZMQ.jl is kept the same) and then running eg
$ julia --project=/path/to/project /path/to/project/bin/push_script.jl
And
$ julia --project=/path/to/project /path/to/project/bin/pull_script.jl
Should do the trick? (Disclaimer: I don’t actually know anything about ZMQ or what you’re trying to do, so I may be missing something fundamental)
1 Like
Oh, re-reading your initial post, you should also keep in mind that pkg> generate is really for making packages - the relationship between projects and packages is a little confusing, but it sounds like you could just start with
$ mkdir my_zmq; cd my_zmq
$ julia
julia> ]
1.8> activate .
my_zmq> add ZMQ
#...
# profit
Rather then having all the rigamarole of a package
first of all thank you for taking the time, really helps.
Secondly I agree I have NOT being at all helpful in helping others to help myself. Sorry about that. ZMQ is a messaging system which allows programs to communicate with each other in structured manner. I use it in this manner.
script Pone.jl generates a message which has the following form <1000><3433.343> it posts it to ZMQ which sends it to script Pthree.jl which dequeues it from ZMQ parses the message and populates a dataframe with the information in the message.
script Ptwo.jl generates a message which has the following form <2000><3433.343> it posts it to ZMQ which sends it to script Pthree.jl which dequeues it from ZMQ parses the message and populates a dataframe with the information in the message.
So ZMQ is my way of allowing me to distribute the processing. I start Pthree.jl using cron and it just sits there waiting on port 5555 for messages. Cron starts Pone.jl which starts to post messages TO port 5555 and so does Ptwo.jl.
this allows me to control the timing of the message flows, create more data feeds ( to 5555) and Pthree ( the pull server) can handle them easily.
Hopefully this shows the problem.
Hi all
thanks so much to @kevbonham for trying to help me. I decided that I hadn’t done a great job framing the question so I tried again here.
I tried to provide more information and a mwe.
thanks again @kevbonham you are what makes this community great.