# Memory issues for large computational graph

**URL:** <https://discourse.julialang.org/t/memory-issues-for-large-computational-graph/112606>\
**Category:** Modelling & Simulations\
**Tags:** memory-allocation, sciml, enzyme\
**Created:** [April 6, 2024, 2:25pm UTC](https://discourse.julialang.org/t/memory-issues-for-large-computational-graph/112606 "2024-04-06T14:25:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arijit\_Bhattacharya](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arijit_bhattacharya/32/207717_2.png) [@Arijit\_Bhattacharya](https://discourse.julialang.org/u/Arijit_Bhattacharya)\
**Post date:** [April 6, 2024, 2:25pm UTC](https://discourse.julialang.org/t/memory-issues-for-large-computational-graph/112606/1 "2024-04-06T14:25:21Z")

</div>

I’m working on developing a differentiable CFD solver incorporating the immersed boundary method for solving the Navier-Stokes equations. I plan on using Enzyme.jl .My goal is to compute gradients of a parametric geometry with respect to a custom-defined loss function, which I plan to optimize using gradient descent. However, I’m encountering memory issues due to the large computational graph resulting from multiple iterations (i.e., time steps). Are there any effective strategies to mitigate this problem?

Thank you in advance for your suggestions!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [April 6, 2024, 2:39pm UTC](https://discourse.julialang.org/t/memory-issues-for-large-computational-graph/112606/2 "2024-04-06T14:39:39Z")

</div>

For time stepping adjoints, you normally want to solve this by not differentiating the solver directly but instead define some adjoints over it that optimize a few things. Some of things you can optimize are memory with continuous checkpointing and compute time because some of the computations can be skipped. If you use DifferentialEquations.jl for the time stepping or NonlinearSolve.jl for the nonlinear solving these optimizations will be automatically applied.

---

<div class="post-metadata">

**Author:** ![Arijit\_Bhattacharya](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arijit_bhattacharya/32/207717_2.png) [@Arijit\_Bhattacharya](https://discourse.julialang.org/u/Arijit_Bhattacharya)\
**Post date:** [April 6, 2024, 3:05pm UTC](https://discourse.julialang.org/t/memory-issues-for-large-computational-graph/112606/3 "2024-04-06T15:05:44Z")

</div>

Could you suggest some resources, I could use to learn about time-stepping adjoints for PDE?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [April 6, 2024, 3:11pm UTC](https://discourse.julialang.org/t/memory-issues-for-large-computational-graph/112606/4 "2024-04-06T15:11:56Z")

</div>

[![](https://global.discourse-cdn.com/julialang/original/3X/a/7/a7306bcec5cd546dc07d53639a27d450e71942af.jpeg "Introduction to Scientific Machine Learning in Astroinformatics Part 2: Numerics") ](https://www.youtube.com/watch?v=rVMqjW6CKmw)

That’s probably the most comprehensive these days, though it’s missing some of the newer tricks.

> **[A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis...](https://ieeexplore.ieee.org/abstract/document/9622796)**
>
> Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be difficult...

is also an overview, but missing some of the newer optimizations. The SciMLSensitivity.jl source code really describes it in full 😅