# RFC: Reverse Communication (ask/tell) adapter for function-style optimizers in Julia?

**URL:** <https://discourse.julialang.org/t/rfc-reverse-communication-ask-tell-adapter-for-function-style-optimizers-in-julia/134803>\
**Category:** Optimization (Mathematical)\
**Tags:** optimization, sciml\
**Created:** [December 30, 2025, 1:38pm UTC](https://discourse.julialang.org/t/rfc-reverse-communication-ask-tell-adapter-for-function-style-optimizers-in-julia/134803 "2025-12-30T13:38:17Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![karei](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/karei/32/214809_2.png) [@karei](https://discourse.julialang.org/u/karei)\
**Post date:** [December 30, 2025, 1:38pm UTC](https://discourse.julialang.org/t/rfc-reverse-communication-ask-tell-adapter-for-function-style-optimizers-in-julia/134803/1 "2025-12-30T13:38:17Z")

</div>

Hi all,

I’d like to start a discussion on a generic Reverse Communication Interface (RCI), i.e. an ask/tell-style session API, for Julia’s optimization ecosystem. The idea is to sit on top of existing “function-style” optimizers (where the solver owns the loop and calls `f(x)` internally).

### Motivation

A lot of optimizers in Julia (including many wrappers) follow the usual pattern:

- you provide `f(x)` (and maybe constraints `g(x)`),
- the solver runs the loop and repeatedly calls `f(x)`.

That’s perfect for in-process optimization. But it gets awkward when objective evaluations are expensive or externally managed, e.g.:

- HPC job queues / distributed simulation pipelines
- lab experiments or licensed simulators
- strict evaluation budgets, retries, or failure handling

In these cases, it’s often much easier if the solver can be driven via something like:

- `x = ask!(session)`
- evaluate externally (possibly async)
- `tell!(session, x, f, g...)`

This “reverse communication” pattern shows up in a lot of numerical libraries.

### Prior art

Nevergrad (Python) has a “recaster” subsystem that wraps function-style third-party optimizers into ask/tell by running the solver in the background and proxying objective calls through a message channel (`recastlib.py` / `recaster.py`). Not suggesting we copy it, but it’s a concrete example of the pattern.

### Proposal (discussion-level)

A small, generic adapter layer that can “recast” function-style solvers into an RCI session:

- minimal API: `ask!`, `tell!`, `done`, `result`
- a clear lifecycle: cancellation / shutdown to avoid deadlocks
- could live in Optimization.jl, or as an experimental add-on first (e.g. `OptimizationRCI.jl`)

### Limitations / expectations

- This won’t magically make sequential solvers batch-parallel. Many will still request one evaluation at a time.
- The main value is externalizing evaluation and plumbing (budgeting, caching, job scheduling, fault tolerance), not solver speedups.
- Backend coverage would likely be incremental; some wrappers may be tricky due to threading/FFI constraints.

Would love to hear thoughts, references, or “this is a bad idea because …” feedback.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [December 30, 2025, 7:44pm UTC](https://discourse.julialang.org/t/rfc-reverse-communication-ask-tell-adapter-for-function-style-optimizers-in-julia/134803/2 "2025-12-30T19:44:02Z")

</div>

My suggestion if you’re interested in this is just to prototype it!

Write it three different ways. Try it out. It’s a lot easier to have a discussion about concrete implementation than an abstract idea.