Hello everybody,
I’m currently working on a project to “solve” the patient-bed assignment problem in emergency department (ED) of hospitals using agent-based modelling and reinforcement learning. I’m very new to using Julia and don’t have a lot of knowledge coding and therefore I’m looking for online tutoring (e.g. Zoom) for my specific project and am grateful for any kind of help.
A simple breakdown of my project would be e.g.:
I have an environment/space of 4 rooms/beds with different properties. A patient enters the ED with a specific level of illness (1-4). Level 1 would be almost not sick at all and can use any free bed because he has no real requirements to be treated, and level 4 would be extremely sick, who can only use the one bed which is best suited for a level 4 illness. Now the bed assigner (agent?) must assign the patient to a free bed.
The idea is to create & compare three different models and evaluate the results.
The first model should be a random assignment. The patient, regardless of its level of illness will be randomly assigned to any of the free beds.
The second model should use an intelligent logic/heuristic. For example, something like: If a patient of level 4 arrives, assign him to bed number 4 if it is free. Else assign him to the best possible alternative.
The third model should use reinforcement learning to improve to assignment of the patients to the beds.
Furthermore, I would love to visualize the patient-flow/assignment. I’m imagining something like a simplified map of the ED looked from above with paths that the patients are following to the beds. The patients should have different colours depending on their state (e.g., red for not in a bed, green for in a bed and black for leaving). I have drawn a little example:
I have already done some research regarding the best possible packages to use for my project and the packages that left the best impression are “Agents.jl” and “POMDPs.jl”.
I really like the visualisation capability of both and the coding structure of Agents and the MDP approach for the agents from POMDPs but I don’t know how to combine both or if it would even be possible/advised for my case.
The problem that I’m facing is that I didn’t find a good example-model to look at the code and use it as inspiration and therefore don’t know how to write my agents with action, state and rewards and my space/environment… I don’t know how to start.
If anybody knows how to help me and would be willing to give me tutoring lessons (of course with payment) over Zoom I would be super happy.
Also, I’m grateful for any tips and explanations.
Greetings
Floyd