I spend some time talking with @davide445 and below I wrote summary of his problem.
Final aim is to develop simulator of surgery navigation for educational purpose, that can be interacted with by using haptics device. It is easier to show it than explaining by words, so check this video.
For this simulator be useful it must work in real time, rendering 60 frames per second (60 FPS), with roughly 30 ms per frame. You need to ask @davide445 why this 30 ms is so important, I don’t know that. Today this can be done for simulations when the main interest is operation on bones, because bones can be treated in large extent as rigid bodies (no surprise here). But making a simulation of soft tissue e.g. liver is much more challenging computationally. Again, no surprise.
Program must works in real time and must simulate cutting of soft tissue in the way that is realistic enough.
Program of simulation roughly operate in the way listed blow. @davide445, I hope I get it mostly right.
- Take data of liver and produce it visual model.
- Take user input.
- Produce data input for PDEs that describe movement of the liver.
- Send PDEs data input to solver.
- Solve PDEs with given PDEs data input, generated PDEs data output.
- Send PDEs data output from solver to visualisation of solver.
- Go back to point 1.
Main question of @davide445 is: should he give Julia and SciML ecosystem as computation engine in point 5? In other words, we can assume that visualisation and I/O software is ready and we want use Julia only for computing time evolution of liver. Is this worth a try?
The goal is to find such a way to make such simulation possible under time constrain of 30 ms, while only some part of this time can be used for solving problem.
My intuition is: yes, Julia and SciML can give you as good performance in solving differential equations as you can get. If some solver is faster, you just open GitHub issue “Hey, SciML solver is slower than solver X.” and then collaboration will work until they remove this bug (this is how Chris Rackauckas describe their workflow in one talk from 2020). But there are few caveats.
First caveat. From what I understand @davide445 and his collaboration still aren’t sure if solving PDEs isn’t to big bottle neck and in the end some Machine Learning is needed to make computation fast enough. I trust SciML when you need to solve differential equations and never check they ML ability. I’m also agnostic about how good SciML is in mixing PDEs solvers with ML algorithms, I never put too much attention to neural differential equations.
Second caveat. In point 4 and 5 you need pipe Julia code with external software, this can be a big bottle neck. From what @davide445 told me, they have their own program that works entirely on GPU, where CUDA is used for communication, sharing memory and all other things. I’m not GPU guy, so I may understand him wrong and I can’t judge if would it be a problem or not.
At this stage I don’t know how much time implementing such thing in Julia would take. It can be work for 3 days or for 3 months and @davide445 need opinion someone knowledgeable is this effort worth pursuing or not? I can’t give him such answer and he need it to the end of the April at most.
