# Physics-informed training of a surrogate model involving finite element analysis

**URL:** <https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735>\
**Category:** Machine Learning\
**Tags:** machine-learning, finite-element, sciml\
**Created:** [September 11, 2023, 9:43am UTC](https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735 "2023-09-11T09:43:49Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![touste](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touste/32/10758_2.png) [@touste](https://discourse.julialang.org/u/touste)\
**Post date:** [September 11, 2023, 9:43am UTC](https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735/1 "2023-09-11T09:43:49Z")

</div>

Hi all,

I’m looking at ways to accelerate mechanical simulations for endovascular applications. We published a paper recently where we present [EndoBeams.jl](https://www.sciencedirect.com/science/article/pii/S0965997822000849), a finite element framework for beam elements in contact with a vessel in Julia.

We are now working on accelerating these simulations, given a parametrization of the vessel and of a stent. We are using [Surrogates.jl](https://github.com/search?q=Surrogates.jl&type=Repositories) to sample and simulate many deployment cases and build a surrogate model. The output we are interested in is the vector of nodal positions of a deployed stent. We have had some success using Gaussian Processes, but the number of training cases required is huge (we have ~12 parameters, and one simulation takes ~1h).

As a consequence, I would like to explore training a neural network as a surrogate, and adding to the loss function terms that penalize out-of-balance mechanical equilibrium, i.e. the residual of the FE simulation in the final configuration. I suspect this is similar to what PINNs are doing, except in the context of a surrogate model here. I expect that by enforcing the physical laws of the system, it will regularize the training, increasing the accuracy of the surrogate model even with a low number of training cases.

If you’ve followed me so far, I have a few questions. Bear in mind that I’m a mechanical engineer and lack a bit of knowledge in machine learning, but I’m willing to try and learn!

- Does that sound like a valid approach?
- Is there an example somewhere of a similar approach that could get me started?
- I suspect that if add terms to my loss function, I will need to provide gradients for the training. I should be able to make my FE simulation differentiable using the adjoint method for example. Should I use a `rrule` to provide the gradients to Flux? Or can I use something like [ImplicitDifferentiation.jl](https://github.com/gdalle/ImplicitDifferentiation.jl)?

Sorry if this is more a scientific problem than a question about the Julia language itself, I hope this is appropriate here.

Any help will be appreciated!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 11, 2023, 8:19pm UTC](https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735/2 "2023-09-11T20:19:08Z")

</div>

> [@touste](#):
>
> I suspect this is similar to what PINNs are doing, except in the context of a surrogate model here. I expect that by enforcing the physical laws of the system, it will regularize the training, increasing the accuracy of the surrogate model even with a low number of training cases.

You can even mix the two, which can be good or bad depending on the circumstance.

> [@touste](#):
>
> Is there an example somewhere of a similar approach that could get me started?

I don’t know what the best tutorial would be but if you’re interested in using JuliaSimSurrogates this tutorial pretty much hits the nail on the head.

> **[404 · JuliaHub](https://help.juliahub.com/juliasimdigitalecho/dev/examples/bruss/)**
>
> Documentation for JuliaHub.

Generating a surrogate of a nonlinear PDE model for the purpose of accelerating future parameter estimation tasks.

> [@touste](#):
>
> I suspect that if add terms to my loss function, I will need to provide gradients for the training. I should be able to make my FE simulation differentiable using the adjoint method for example. Should I use a `rrule` to provide the gradients to Flux? Or can I use something like [ImplicitDifferentiation.jl](https://github.com/gdalle/ImplicitDifferentiation.jl)?

If you use NonlinearSolve.jl for the solving, it’ll specialize on sparsity and handle the `rrule`, so there’s nothing to worry about there.

---

<div class="post-metadata">

**Author:** ![touste](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touste/32/10758_2.png) [@touste](https://discourse.julialang.org/u/touste)\
**Post date:** [September 13, 2023, 7:27am UTC](https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735/3 "2023-09-13T07:27:59Z")

</div>

Thanks for the detailed answer! I’ll try on a simple dummy case first to see if everything interfaces properly.

I’m not using NonlinearSolve.jl, I have to use a custom solver to maximize performance (and because updating the state of these beams in a Newton-Raphson scheme is a bit finicky and requires a lot of custom steps). But I’m a bit familiar with how NonlinearSolve.jl handles the custom `rrule` so I should be able to adapt it to my case. Not sure about the sparsity though but I’ll have a look.

I’ll eventually report here if something is not interfacing properly.

---

<div class="post-metadata">

**Author:** ![ksamtsak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ksamtsak/32/10585_2.png) [@ksamtsak](https://discourse.julialang.org/u/ksamtsak)\
**Post date:** [January 19, 2024, 9:24am UTC](https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735/4 "2024-01-19T09:24:13Z")

</div>

@touste this is an interesting problem actually. Did you have any luck since September?

---

<div class="post-metadata">

**Author:** ![touste](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touste/32/10758_2.png) [@touste](https://discourse.julialang.org/u/touste)\
**Post date:** [February 2, 2024, 10:29pm UTC](https://discourse.julialang.org/t/physics-informed-training-of-a-surrogate-model-involving-finite-element-analysis/103735/5 "2024-02-02T22:29:53Z")

</div>

Hi, sorry for the late reply. Not much progress unfortunately but I’m still thinking about it!