# Looking for Examples of Symbolic Regression for predicting Hidden Markov Models

**URL:** <https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749>\
**Category:** Machine Learning\
**Created:** [December 26, 2025, 2:49pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749 "2025-12-26T14:49:58Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![mj2984](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mj2984/32/43833_2.png) [@mj2984](https://discourse.julialang.org/u/mj2984)\
**Post date:** [December 26, 2025, 2:49pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/1 "2025-12-26T14:49:58Z")

</div>

I have a set of time series data that track an observable _(o(t))_ vs control signal _(c(t))_ over a period of time. One way I know to model this system is using Recurrent Neural Networks or some variant of it. I am looking to find a simplified inferable model that I could use in a control loop (less computationally expensive than RNN).

If my system state is _s(t)_, the function **f\_observermap(s(t)) = o(t)** maps from state to observable, and the function **f\_update(s(t),c(t)) = s(t+1)** updates the state based on control signal.

I am trying to learn s, f\_observermap and f\_update as simplified expressions.

A simplified case is s(t) = o(t) i.e. there is no real hidden state and the memory is fully characterized by o(t), which would make f\_observermap(s(t)) = s(t) and the only thing left to figure out is f\_update(o(t),c(t)) = o(t+1). In this case, I can create pairwise data for each (t,t+1) timestamps and learn f\_update using symbolic regression.

Is there a way to have symbolic regression learn the general case where f\_observermap and s is unknown but the data is simple enough to learn a short expression. In the general case multiple distinct states _(s1 != s2)_ may have same observable i.e. _f\_observermap(s1) = f\_observermap(s2)_, but _f\_update(s1,c) != f\_update(s2,c)_. Alternately, what other options do I have other than symbolic regression for this purpose.

---

<div class="post-metadata">

**Author:** ![mj2984](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mj2984/32/43833_2.png) [@mj2984](https://discourse.julialang.org/u/mj2984)\
**Post date:** [December 27, 2025, 12:16pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/2 "2025-12-27T12:16:10Z")

</div>

I think I have some way to formalize my problem. It can be thought of as finding 2 equations instead of 1. Wonder if there’s a way to try and co optimize two equations.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [December 27, 2025, 12:34pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/3 "2025-12-27T12:34:52Z")

</div>

Maybe you could try combining HiddenMarkovModels.jl with SymbolicRegression.jl?  
See this example for a simple linear regression setting: [Control dependency · HiddenMarkovModels.jl](https://gdalle.github.io/HiddenMarkovModels.jl/stable/examples/controlled/)

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [December 27, 2025, 2:17pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/4 "2025-12-27T14:17:50Z")

</div>

Are you in the Gaussian world or the discrete world? In principle this can be done, compare [Package interoperability showcases - #8 by mschauer](https://discourse.julialang.org/t/package-interoperability-showcases/33734/8)

---

<div class="post-metadata">

**Author:** ![mj2984](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mj2984/32/43833_2.png) [@mj2984](https://discourse.julialang.org/u/mj2984)\
**Post date:** [December 27, 2025, 4:11pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/5 "2025-12-27T16:11:14Z")

</div>

I would say more discrete than gaussian because my control signals are uniformly spaced and the span forms fairly discrete patches (these patches themselves are somewhat gaussian though). It’s similar to Newtonian motion physics problems (based on say position and time, it has to predict acceleration to be the rule for governing equation). The problem is I am unsure how much amount of history is encoded at a given state (for the Newtonian motion, it is two-time steps, but for mine I’m unsure).

Currently I am doing a brute force approach, refactoring my inputs to contain h previous history. If the number of observations is _O_ₜₒₜₐₗ and for each observation total observation timesteps is _T_ₜₒₜₐₗ, with the first h timesteps having no control signal, then I have

```julia-auto
Observations = rand(Float32,Tₜₒₜₐₗ,Oₜₒₜₐₗ)
ControlSignals = rand(Float32,Tₜₒₜₐₗ-h,Oₜₒₜₐₗ).

```

I define my input for the regressor as

```julia-auto
TotalDataPoints = (Tₜₒₜₐₗ-h)*Oₜₒₜₐₗ
Input = Array{Float32}(undef,h+1,TotalDataPoints)
for historyOffset in range(0,h-1)
    Input[historyOffset+1,:] = reshape(Observations[historyOffset+1:historyOffset+TotalDataPoints,:],TotalDataPoints)
end
Input[h+1,TotalDataPoints = reshape(ControlSignals,TotalDataPoints)
Output = reshape(Observations[h+1:h+TotalDataPoints,:],TotalDataPoints)

```

Then I ask the model to fit an expression from Input to Output. This gives me an expression f(x₁,x₂..xₕ,xₕ₊₁) where x₁,x₂…xₕ are the Observations up to h history and xₕ₊₁ is the current Control signal.

Also, how do I customize the loss function for Symbolic Regression. I would like the worst-case error to be minimum instead of mean error (something like maxabs), since the dataset represents many real worlds use patterns and we would like to minimize worst case patterns.

Thank you for the suggestion on Hidden Markov models. I’ll try to explore it.

---

<div class="post-metadata">

**Author:** ![mj2984](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mj2984/32/43833_2.png) [@mj2984](https://discourse.julialang.org/u/mj2984)\
**Post date:** [December 27, 2025, 6:53pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/6 "2025-12-27T18:53:24Z")

</div>

I’m curious if it would be possible for Symbolic Regression to automatically learn powers (like adidabatic process for ideal gas PV^(γ) = constant). Adding ^ to unary operator candidates throws an error. Is there a way to make this work better and be more generic?

I tried writing my own custom function, for example

```julia-auto
my_square(x) = x^2

```

but I am unable to get it working for fractional powers

```julia-auto
pow_onepointfive(x) = x^(1.5)

```

Since it has domain error for x \< 0. In the list of functions I have used, log also has similar domain, but it works fine. Is there a way to enforce additional constraints, or relaxations to make it work.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [December 27, 2025, 8:42pm UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/7 "2025-12-27T20:42:44Z")

</div>

@MilesCranmer

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [December 30, 2025, 2:53am UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/8 "2025-12-30T02:53:01Z")

</div>

Definitely check out TemplateExpressions in SymbolicRegression.jl, they are a way to customize your learning problem into an arbitrary structure. For example in the following example we learn a time-dependent magnetic field and drag function: [https://ai.damtp.cam.ac.uk/symbolicregression/v1.12.0/examples/template\_expression](https://ai.damtp.cam.ac.uk/symbolicregression/v1.12.0/examples/template_expression)

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [December 30, 2025, 2:54am UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/9 "2025-12-30T02:54:39Z")

</div>

And [Parametrized Template Expressions | SymbolicRegression.jl](https://ai.damtp.cam.ac.uk/symbolicregression/v1.12.0/examples/template_parametric_expression) for how to include a vector of learnable parameters.

---

<div class="post-metadata">

**Author:** ![mj2984](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mj2984/32/43833_2.png) [@mj2984](https://discourse.julialang.org/u/mj2984)\
**Post date:** [December 31, 2025, 2:03am UTC](https://discourse.julialang.org/t/looking-for-examples-of-symbolic-regression-for-predicting-hidden-markov-models/134749/10 "2025-12-31T02:03:53Z")

</div>

This is Fantastic. Thanks for all the work. I am going to explore this now.