# Difference-Differential system in ModelingToolkit.jl

**URL:** <https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605>\
**Category:** Numerics\
**Created:** [October 19, 2020, 2:07am UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605 "2020-10-19T02:07:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Honza9723](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/honza9723/32/7807_2.png) [@Honza9723](https://discourse.julialang.org/u/Honza9723)\
**Post date:** [October 19, 2020, 2:07am UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605/1 "2020-10-19T02:07:55Z")

</div>

Dear All,

I would like to ask for advice regarding ModellingToolkit.jl. I am working on a problem (macroeconomic model) that reduces to the solution of a highly nonlinear dynamic system. Because of high-nonlinearity and lack of theoretical results regarding properties of the solution to that system, I would like to solve the system using a neural network as @ChrisRackauckas recommended here many times for “weird” problems. I looked at some examples/tutorials of ModelingToolkit.jl, and I like its syntax and ability to autoparallelize on GPUs really a lot.

However, my system is a mix of nonlinear difference/recurrence equations with differential equations, some parts contain integral (expectation) with respect to some random variables and these dynamic equations are coupled by few algebraic/static equations.

So, my question is, whether ModelingToolkit.jl could handle this type of system, or I had to write something on my own. If it is the case that ModelingToolkit.jl can’t handle this type of system, I would be grateful for any suggestion for some packages/frameworks that could substitute Toolkit, ideally as idiot-proof and parallel as possible.

As a second question, I would like to ask, whether there is some easy way, how to parallelize this type of code on TPUs, I saw some examples of Flux.jl running on TPUs, but I am not sure how robust/easy to use it is.

Thanks in advance!  
Honza

---

<div class="post-metadata">

**Author:** ![Honza9723](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/honza9723/32/7807_2.png) [@Honza9723](https://discourse.julialang.org/u/Honza9723)\
**Post date:** [October 21, 2020, 1:17am UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605/2 "2020-10-21T01:17:54Z")

</div>

Hi. Any guidance about difference equations in ModelingToolkit? Is ModelingToolkit able to handle difference equations or mixed difference-differential systems?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 24, 2020, 6:29am UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605/3 "2020-10-24T06:29:02Z")

</div>

It’s not implemented yet but I’m hoping to try and get there ASAP. Differential-difference equations would likely not be very good on TPUs since BFloat16 numbers won’t be sufficiently accurate for most control systems.

---

<div class="post-metadata">

**Author:** ![Honza9723](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/honza9723/32/7807_2.png) [@Honza9723](https://discourse.julialang.org/u/Honza9723)\
**Post date:** [October 24, 2020, 9:02pm UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605/4 "2020-10-24T21:02:01Z")

</div>

@ChrisRackauckas Thank you very much for your reply! I will try to fiddle with my own code in the meantime. So TPU-training of the network for an approximation of the solution function of my system isn’t a good idea? What about GPU, do you think that there is potential for GPU acceleration of training or had to do it on CPUs?

Regarding my system, it is time-independent problem (typical in macro) whose dynamics is captured just by state variables, and those variables live on a compact subset of R^n. However, the system is quite nonlinear and includes the differential component.

Best,  
Honza

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 25, 2020, 4:55am UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605/5 "2020-10-25T04:55:08Z")

</div>

> [@Honza9723](#):
>
> So TPU-training of the network for an approximation of the solution function of my system isn’t a good idea?

Doing it via a physics-informed neural network could work if the stiffness is sufficiently low. GPU is fine though: Float32 is a lot easier to handle than BFloat32.

> [@Honza9723](#):
>
> Regarding my system, it is time-independent problem (typical in macro) whose dynamics is captured just by state variables, and those variables live on a compact subset of R^n. However, the system is quite nonlinear and includes the differential component.

That does seem like something to try a PINN on, and it’s something we’re looking into.

---

<div class="post-metadata">

**Author:** ![Honza9723](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/honza9723/32/7807_2.png) [@Honza9723](https://discourse.julialang.org/u/Honza9723)\
**Post date:** [October 26, 2020, 7:34pm UTC](https://discourse.julialang.org/t/difference-differential-system-in-modelingtoolkit-jl/48605/6 "2020-10-26T19:34:10Z")

</div>

@ChrisRackauckas Thank you very much for your guidance!

Best,  
Honza