Hi there
we have had a goal of
1 getting a real time data feed to update a data structure in a reliable manner.
2 use a notebook UI to effect the graphical representation of the the datastructure contents
3 allow 1 - 10 people to use the UI to make decisions.
We , thanks to this wonderful community, got 1 working very quickly. 2 is somewhat problematical but works once we throttle the feed.
we have hit a stumbling block with 3.
our approach “was” to use ZMQ to feed a PlutoHOOKs ( all hail @Pangoraw ) cell and thus maintain a datastructure in another PlutoNoteBook cell. We were going to use PlutoUI and DataFramesMeta to curate another dataframe which would be used to make decisions. We were going to host the notebook on AWS and use PlutoSliderServer to provide user independence of action. We have been advised that PlutoSlider server doesn’t work with PlutoHooks therefore our approach will not work.
We don’t use github so any solution using that approach is not something we wish to consider. We have a Stipple.jl approach for web serving working but the datastream keeps freezing and no one seems to know why, we did ask in the forums but nothing came of it.
We aren’t married to Pluto but we do love it, ( all hail @fonsp ) we can work with any solution that ends up with user manipulation of a real time updated datastructure ( right now we are using dataframes) which can be deployed on a webpage. Again we love DataFrameMeta all hail @bkamins
We would like to know if anyone has an approach we can consider to achieve goal 3. We have goal 1 working well, goal 2 is on hold until we get an answer to goal 3.
Please note we just want the HIGH level approach not HOW to do it. We’re not looking for code, just the approach.
We are confident that we can get all of the goals working in python, dash and pandas within a very short space of time We have been toying with julia for some time now and committed the month of April 2023 to all julia, happy to extend for a couple of weeks into May but after that we have to make a decision.
What I would do for 3. is
- upload the notebook in a server
- start julia and pluto on the server
- set up some port forwarding to these 10 people and so that they can access the notebook from their browser.
I hope I am on the right direction and I didn’t misuderstand what you want here.
If all these 10 users need separate notebook/UI states, you can simply run 10 independent Pluto notebook instances.
first thank you for taking the time. We did look into this using PlutoSliderServer but PlutoHooks doesn’t work with it. We need the realtime feed in a cell which other cells feed off. Thanks for the suggestion though.
I get the idea and it would be simple to set up and maintain using ZMQ and cron. Certainly worth considering. Thanks for the suggestion.
Can you please explain why do you need PlutoSliderServer ? Is it something that is planned to be publicly available or just for a limited team ?
certainly. It’s for a small team 4 -6 people. 2 remote ( hence the web req. We don’t want to expose our network to the outside thus running a server in house and giving the remote access won’t work for us. We transform the data in house and pump it out down a curated ZMQ stream to the dataframe running on the server outside the building. We just wanted to represent the dataframe in Pluto and transform it using widgets.
We LOVE the utility of Pluto thus PlutoSliderServer is a must if we want to use PlutoUI widgets, which we do. We also want to update a datastructure real time ( trimmed to 1 second intervals) on which the PlutoUI widgets would be used to transform it. Probably using the splendid DataFramesMeta from @bkamins .
we have tried Stipple ( fine product ) but the data feed freezes on screen randomly.
We looked at Heroku but hit the same problems as doggo and it means that there might be another set of issues to overcome in Pluto but we didn’t get that far.
seems that we don’t have any viable options with if we want to combine julia,Pluto and real time data feed on the web. We just wanted to make sure we weren’t missing anything simple. Thank you so much for your suggestions, we did look into them straight away.