### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [December 21, 2023, 5:05pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/1 "2023-12-21T17:05:43Z")
Hello,
I am using MTK to solve a system of algebraic nonlinear equations. Using `structural_simplify()` helps make these types of problems much easier to solve.
However, I have noticed an inefficiency when trying to return the values of ALL model variables given a simplified solution which only contains values of a reduced number of variables (at least using the only method I am currently aware of).
I have posted a more in-depth description of my issue here. Opening it up here for more viewers to see.
>
>
> \*\*Question❓\*\*
>
> Hello,
>
> I have a question about whether or not a certain fea…ture exists and if not, am making a request for it.
>
> In my example, I am solving a \`NonLinearProblem\`. I use \`structural\_simplify\` to reduce a large list of equations to just a few equations (or none if the system can be solved in sequence without iteration). This is extremely helpful because it reduces the number of initial guesses required and makes solving faster and more robust.
>
> \`\`\`
> \#eqs, components, guess defined elsewhere
> mtk\_model = NonlinearSystem(eqs, \[\], \[\]; systems=components, defaults=guess, name=Symbol(test\_sys))
> mtk\_alias = alias\_elimination(expand\_connections(mtk\_model))
> mtk\_simp = structural\_simplify(mtk\_alias; allow\_symbolic=true, allow\_parameter=true)
> prob = NonlinearProblem(mtk\_simp, \[\])
> sol = solve(prob, NewtonRaphson())
> \`\`\`
>
> In my example, let's say I have a system of 3 components connected one after another in sequence starting from an inlet boundary condition. In this case, this system is trivial and requires no iteration because the list of equations can be computed sequentially given the inlet boundary condition.
>
> Let's say this system has 100 variables given by \`states(mtk\_model)\`, then 50 variables given by \`states(mtk\_alias)\` and 0 variables given by \`states(mtk\_simp)\` (because in this example, no iteration is technically required although this same concept is still applicable if I connected the components differently such that iteration was required). The obvious choice is to solve with \`mtk\_simp\` because it's easier to solve and does not require providing 50 to 100 guesses. Then, \`sol.u\` will return a \`Vector{Float64}\` corresponding to the 0 variables leftover. Of course, I still want to know the variable values of the other 50 or 100 values. I know I can return these values by doing this:
>
> \`\`\`
> vault = \[\]
> for s in states(mtk\_alias)
> val = sol\[s\]
> vault = vcat(vault, val) # or insert other code to store the value after each iteration
> end
> \`\`\`
> However, I have found this method to be very inefficient, because this action re-computes all of the equations from the inlet boundary condition to the equation in a downstream component that ultimately provides the value of the requested variable \`s\`. This is problematic for my case because I have some non-symbolic "expensive" function evaluations in my equation list. This method ends up dramatically blowing up the run time required to see all variable values. I have found this issue to to be less critical in instances where the function evaluations are less expensive but the process is still redundant, nonetheless.
>
> An analogy I like to use to represent this case comes from a classic basketball conditioning exercise. There is a drill to run from baseline to free throw line and back, then to half court and back, and then to the opposite free throw line and back, and then to the opposite baseline and back. The above method is doing this same thing in terms of equations (i.e., replace court lines on a court with equations in an equation list as we go down the component list in the system). In reality, we only care about going from baseline to baseline (equation 1 to the last equation) and returning all variable values from the sequence of computations along the way. That would be the most efficient way to return all variables values--in one swoop rather than in increments. Does this method exist in MTK or is there a trick that can be coded? Perhaps I am just missing it.
>
> I know another alternative method to reduce the runtime of grabbing values \`sol\[s\]\` is to solve the larger systems like \`mtk\_alias\` or \`mtk\_model\`. In that case, the output \`sol.u\` already stores all the variable values that I want to see; then using \`sol\[s\]\` returns the requested variable value rapidly without having to recompute equations (I have confirmed this). However, I have found that providing large lists of guesses is cumbersome, can affect the robustness of the successful solve if not carefully provided ( it matters alot in my system, maybe not in others), and will take longer to solve. I really would love to find a way to use \`structural\_simplify\` AND find an efficient way to return all variables values. Would help me tremendously! Thanks, Chris
In summary, for an example where I have a model of 100 variables which simplifies down to 5 variables, how can I return the values of all 100 variables given a simplified `sol.u` vector output which only contains 5 variable values? I only currently know to do this: `sol[var6], sol[var7], .... sol[var100]` in a for-loop manner which returns the remaining variable values in a one-by-one manner.
However, from my observation, I have found this method to repeat calculations unnecessarily, and it becomes problematic when the model contains expensive function evaluations (like in my case) which leads to excessively long wait time to get the full solution returned. Is there a way to take the simplified solution and compute all remaining variables in “one swoop” and return all of those variable values? (rather than computing all equations up to the `var10`, then repeating up to `var20`, then repeating up to `var100` and returning each variable value individually)
I know solving an un-simplified equation list of 100 vars and 100 equations will give me a `sol.u` vector of all 100 variables stored in memory. Extracting `sol[var1], sol[var2], ....sol[var100]` has no wait time at all in this case. However, solving the larger system of equations can be more difficult, and it is more cumbersome to provide 100 initial guesses (my system is not as trivial as providing 1.0 as each variable’s guess). The perfect solution is to `structual_simplify` and then be able to compute and return all variable values at once. Is this possible?
Thanks, Chris
---
### Author:  [@contradict](https://discourse.julialang.org/u/contradict)
#### Post date: [December 21, 2023, 6:23pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/2 "2023-12-21T18:23:17Z")
An example might make this discussion a little easier.
```julia
using ModelingToolkit, DifferentialEquations
function test()
@variables t x(t)=1.0 y(t)=0.0 z(t)=0.0 a(t)
@parameters σ=10.0 ρ=26.0 β=8/3
D = Differential(t)
eqs = [
D(x) ~ σ * (y-x)
D(y) ~ x * (ρ - z) - y
D(z) ~ x * y - β*z
a ~ x+z
]
@named sys = ODESystem(eqs, t)
sys = structural_simplify(sys)
prob = ODEProblem(sys, [], (0.0, 10.0))
sol = solve(prob)
end
```
Then you can do something like this to get all values.
```julia
julia> sol = test();
julia> allvars = vcat(states(sol.prob.f.sys), ModelingToolkit.lhss(observed(sol.prob.f.sys)))
4-element Vector{Any}:
x(t)
y(t)
z(t)
a(t)
julia> sol(5.0; idxs = allvars)
4-element Vector{Float64}:
-9.173169284085155
-7.283715989395803
28.46992558909574
19.296756305010586
```
---
### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [December 21, 2023, 8:22pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/3 "2023-12-21T20:22:59Z")
Thanks for the response.
the line `allvars = vcat(states(sol.prob.f.sys), ModelingToolkit.lhss(observed(sol.prob.f.sys)))` returned the list of all variables in my Non-linear system. Does this do the same thing as `states(ODESystem(eqs, t))` in your `test()` func?
Also, for a `NonlinearProblem`, there is no time dependence. I assume `5.0` refers to a timestep in `sol(5.0;idxs=allvars)`. Any idea how you would do it for my case? I tried `sol(;idxs=allvars)` and it failed. Doing `sol[allvars]` seems to repeat the issue I discussed as it is no different than doing the for-loop method to return individual states one at a time.
Thanks, Chris
---
### Author:  [@contradict](https://discourse.julialang.org/u/contradict)
#### Post date: [December 21, 2023, 9:41pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/4 "2023-12-21T21:41:51Z")
Ah, I missed that you had a NonlinearProblem. Yes, the concatenation of the states and the observables should be the same as the list of states of the unsimplified problem.
I think `sol[allvars]` is as good as it gets. In the unsimplified case, you pay the cost of computing the expensive functions while solving, in the solution indexing case you pay it after solving. Is there some structure to exploit among the observed variables like common subexpressions that might help? If there is, you might try
```julia
build_function(ModelingToolkit.rhss(observed(sol.prob.f.sys)), states(sol.prob.f.sys))
```
And see if the compiler can find and optimize that in your case.
---
### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [December 21, 2023, 10:39pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/5 "2023-12-21T22:39:05Z")
I am still puzzled why `sol[allvars]` is as good as it gets. I believe there should be a way to improve this if it doesn’t exist already, because I believe the sentiment that this cost is unavoidable whether you pay “during the solve” or “after the solve” is incorrect for the following reason:
The reason that the cost is exacerbated when doing this “after the solve”, is that `sol[allvars]` is repeating the same expensive calls over and over again as it makes it way through the list of `allvars`. I have been able to confirm this since I can put `@show` statements within my registered functions; the same calls are being done over and over again as you work down the list of `allvars`.
**I am simply looking for a way that once the simplified solution is found, compute ALL equations once from start to finish and store all variable values along the way.** However, `sol[allvars]` will compute the equations required to return the value of `var1`, then the equations required for `var10`, then the same for all equations all the way to `var100` separately; in this case the run time to return `var1` to `var100` gets gradually longer and longer. In a basic example, if you do `sol[var1]` it may be quick, then `sol[var10]` a bit slower, then `sol[var100]` being the slowest. All three of these calls only return 1 variable value, but in order to return the value of `var100`, it had to compute `var1` and `var10` along the way to get to the answer of `var100`. In the end, using `sol[var100]` computes all the variables 1-100 but only returns `var100`; frankly, from an outsider looking in, I don’t see a reason why it has to be this way (unless I am missing something)
In the link with my github issue, I give an analogy to explain the concept that this is like the basketball training drill where you run from baseline to free throw line and back, then to half court and back, then to the opposite free throw line and back and then to the opposite baseline and back. The `sol[allvars]` is doing this in terms of equations and variables. It feels like there’s no reason it needs to do this; all we care about is going from start to finish in one go.
Now, I realize this is not problematic when expensive functions are not present, so in certain variations of my models, I do not experience huge slow downs, but in other variations where I cannot avoid the expensive function, this aformentioned behavior becomes an issue.
Do you agree or disagree that my observations are correct? Anything I might be missing? Do you think there may be a way to accomplish my proposition of returning all variables at once?
P.S.
Not to be a debbie downer. I want to say that MTK has worked extraordinarily for me to easily build and solve large complex systems of equations for cases with quick function lookups! However, it has been problematic these other very slow cases, but I am hopeful for a fix like the one I suggested. It will help me tremendously. Thanks.
---
### Author:  [@contradict](https://discourse.julialang.org/u/contradict)
#### Post date: [December 21, 2023, 11:01pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/6 "2023-12-21T23:01:15Z")
You are describing a very particular structure of problem here with specific dependencies among variables. I think a runnable example that illustrates your problem (including slowdown with increasing variable index) would be much more valuable and more likely to lead to a solution than a sports analogy. There may be an optimization that can be made in your case, but it sounds like it is not something that would be exploitable in a general problem.
---
### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [December 22, 2023, 4:01pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/7 "2023-12-22T16:01:15Z")
I agree sharing a reproducible code snippet would be ideal, but unfortunately that is difficult for me. It would require a full repository setup due to the integration work we did to implement `CoolProp` thermal lookup properties (the slow culprits). So I’ve only been able to generally describe my case. I could write some pseudo code to explain. I understand this isn’t ideal as I sit here making my plea (haha). I’ll make an attempt at sharing code for a simpler problem that does not require this pre-requisite setup. (will post this separately after)
Regardless, I am still not fully convinced that this behavior is not general. My fundamental question is this: in theory, once you successfully solve the simplified problem, should you be able to compute every remaining variable from the full equation set in a series of computations, given the simplified solution values and your parameters, regardless of the equations?
My understanding is the answer is yes because I have solved a wide range of other more complicated and highly coupled systems where the solution to the simplified solution allows me to to return the value of any variable in the full model. However, they all exhibit this post-processing issue due to the nature of `sol[allvars]`. So I am left wondering: since we can return every variable from the full model, why can we not return them all at once after computing the full list of equations once-through?
---
### Author:  [@contradict](https://discourse.julialang.org/u/contradict)
#### Post date: [December 22, 2023, 5:42pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/8 "2023-12-22T17:42:17Z")
In some cases each variable might be independent and the user may only care about a few, in which case precomputing them all would be just as wasteful.
Did you try the `build_function` approach? I’m not sure it will work, but the idea there is to turn the observed equations into a Julia function which can then be passed to the compiler. The hope is the compiler figures out an evaluation order that takes less work, I’m not even sure it can do that but it is much easier than the next thing I can think of which is using Symbolics.jl to write some custom transformations of the observed equations into a form that caches intermediate values in the way you expect.
Could you create a simplified system with fake data and the same problem structure?
---
### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [December 22, 2023, 8:16pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/9 "2023-12-22T20:16:20Z")
1. I agree that a user does not always care about seeing every variable value; in that case it is wasteful. On the flip side, I think users still frequently want to see many or all variables that were eliminated in the simplification process. Having the option to choose how the solution is post-processed would be a compromise.
2. I haven’t tried that `build_function` option yet. I will likely have to try after the xmas holiday next week.
3. Here is my fake code to mimic what is happening:
```julia
using ModelingToolkit
@connector function TestPort(;name)
sts = @variables T #=p h=# #could add more port variables
NonlinearSystem(Equation[], [sts...;], []; name)
end
function long_func(T, N)
sleep(N) #mimic an expensive function
println("waiting for $N seconds at T=$T")
return 1
#could also execute a longer routine below
#=
sum = 0
for i = 1:N
sum += i
end
return sum
=#
end
@register_symbolic long_func(T, N)::Real
function test_comp(;name, ports, N)
inlet = TestPort(;name=ports[1])
outlet = TestPort(;name=ports[2])
#define params and vars
ps = @parameters N=N
#=sts = @variables s=#
#define equations
#FYI: using an intermediate variable `s` will increase run time of `sol[allvars] (more vars to process)`
eqs = [
#=s ~ long_func(inlet.T, N)=#
outlet.T ~ inlet.T + long_func(inlet.T, N)
#=outlet.p ~ inlet.p
outlet.h ~ inlet.h=#
]
compose(NonlinearSystem(eqs, #=sts=# [], ps; name),
inlet, outlet)
end
function test_boundary_in(;name, ports, val)
port = TestPort(;name=ports[1])
ps = @parameters T=val #=h=val p=val=#
eqs = [
port.T ~ T
#=port.h ~ h
port.p ~ p=#
]
compose(NonlinearSystem(eqs, [], ps; name), port)
end
function test_boundary_out(;name, ports)
port = TestPort(;name=ports[1])
compose(NonlinearSystem(Equation[], [], []; name), port)
end
N1 = 1 #sleep seconds
N2 = 2
N3 = 4
@named baseline = test_boundary_in(; ports=[:out], val=0.0)
@named freethrow = test_comp(; ports=[:in, :out], N=N1)
@named halfcourt = test_comp(; ports=[:in, :out], N=N2)
@named freethrow_opp = test_comp(; ports=[:in, :out], N=N3)
@named baseline_opp = test_boundary_out(; ports=[:in])
eqs = [
connect(baseline.out, freethrow.in)
connect(freethrow.out, halfcourt.in)
connect(halfcourt.out, freethrow_opp.in)
connect(freethrow_opp.out, baseline_opp.in)
]
@named mtk_model = NonlinearSystem(eqs, [], [];
systems=[baseline, freethrow, halfcourt, freethrow_opp, baseline_opp])
mtk_alias = alias_elimination(expand_connections(mtk_model))
mtk_simp = structural_simplify(mtk_alias; allow_symbolic=true, allow_parameter=true)
@time prob = NonlinearProblem(mtk_simp, [])
@time sol = solve(prob, NewtonRaphson(;autodiff=false))
@show sol.u
#return key vars one-by-one
println("--baseline--")
@time sol[baseline.out.T]
println("--freethrow--")
@time sol[freethrow.out.T]
println("--halfcourt--")
@time sol[halfcourt.out.T]
println("--opposite freethrow--")
@time sol[freethrow_opp.out.T]
println("--opposite baseline--") #same time as freethrow_opp because no extra long func
@time sol[baseline_opp.in.T]
#return all vars
println("full system")
allvars = states(mtk_model)
@time sol[allvars]
#estimated wait time for 8 vars in allvars
est = 2*0 + 2*N1 + 2*(N1+N2) + 2*(N1+N2+N3)
println("$est seconds estimated")
#TODO:
#1) repeat for more components in a row and run time will start to inflate for `sol[allvars]`
#2) add more intermediate variables to show `long_func` repetition
```
The run time answers in my terminal are printed below:
```julia
--baseline--
0.001184 seconds (1.31 k allocations: 105.406 KiB)
--freethrow--
waiting for 1.0 seconds at T=0.0
1.033398 seconds (5.46 k allocations: 388.171 KiB, 1.84% compilation time: 100% of which was recompilation)
--halfcourt--
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
3.024640 seconds (5.33 k allocations: 379.195 KiB, 0.54% compilation time: 100% of which was recompilation)
--opposite freethrow--
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 4.0 seconds at T=2.0
7.082794 seconds (5.82 k allocations: 417.484 KiB, 0.51% compilation time: 100% of which was recompilation)
--opposite baseline--
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 4.0 seconds at T=2.0
7.072475 seconds (5.88 k allocations: 421.094 KiB, 0.33% compilation time: 100% of which was recompilation)
full system
waiting for 1.0 seconds at T=0.0
waiting for 1.0 seconds at T=0.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 4.0 seconds at T=2.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 4.0 seconds at T=2.0
22.168350 seconds (11.39 k allocations: 829.844 KiB, 0.23% compilation time: 100% of which was recompilation)
22 seconds estimated
```
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [December 22, 2023, 9:22pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/10 "2023-12-22T21:22:10Z")
> [@chris-hampel-CA](#):
>
> I am still puzzled why `sol[allvars]` is as good as it gets. I believe there should be a way to improve this if it doesn’t exist already, because I believe the sentiment that this cost is unavoidable whether you pay “during the solve” or “after the solve” is incorrect for the following reason:
We’re working on it.
>
>
> \## Checklist
>
> \- \[\] Appropriate tests were added
> \- \[\] Any code changes were …done in a way that does not break public API
> \- \[\] All documentation related to code changes were updated
> \- \[\] The new code follows the
> \[contributor guidelines\](https://github.com/SciML/.github/blob/master/CONTRIBUTING.md), in particular the \[SciML Style Guide\](https://github.com/SciML/SciMLStyle) and
> \[COLPRAC\](https://github.com/SciML/COLPRAC).
> \- \[\] Any new documentation only uses public API
>
> \## Additional context
>
> Add any other context about the problem here.
We just did a complete overhaul of the symbolic indexing interface and one of the things we wanted to achieve was make it so calculation of observed functions could happen in a fused fashion. With that completed, we’re now working to expose some more utilities for large sets of observed equations.
---
### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [January 2, 2024, 4:51pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/11 "2024-01-02T16:51:26Z")
Thanks for the update @ChrisRackauckas. That is great news.
I see that the pull request you linked is merged. Does this indicate that most or all of the required infrastructure is ready such that I can test my script (above) while using a new version of `MTK` and/or `Symbolics`? If so, which released pkg version should I reference?
I would expect in testing my script that `sol[allvars]` will complete in ~7 seconds instead of ~22 seconds, and will report back my finding.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 2, 2024, 6:34pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/12 "2024-01-02T18:34:32Z")
> [@chris-hampel-CA](#):
>
> I would expect in testing my script that `sol[allvars]` will complete in ~7 seconds instead of ~22 seconds, and will report back my finding.
I’m not convinced it’s optimized yet, so there’s that. But I think the interface is done. Right now with the SymbolicIndexingInterface v0.3 release we’re trying to get the interface completed before working on some optimizations. Vectors of observed functions have some pretty massive optimizations which we haven’t look into using yet.
---
### Author:  [@chris-hampel-CA](https://discourse.julialang.org/u/chris-hampel-CA)
#### Post date: [January 11, 2024, 4:51pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/13 "2024-01-11T16:51:29Z")
I re-tested the example script above using the `ModelingToolkit v8.75.0` release, but I am still observing the same behavior as before where it returns each variable value one at a time and repeats calculations along the way.
Is there a different/new function to use to return all variable values that I might not be aware of? And/Or is there still work in progress that I should be waiting on before I re-test this example script?
```julia
julia> @time sol[allvars]
waiting for 1.0 seconds at T=0.0
waiting for 1.0 seconds at T=0.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 4.0 seconds at T=2.0
waiting for 1.0 seconds at T=0.0
waiting for 2.0 seconds at T=1.0
waiting for 4.0 seconds at T=2.0
22.117827 seconds (798 allocations: 90.141 KiB)
```
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [January 11, 2024, 5:46pm UTC](https://discourse.julialang.org/t/how-to-return-all-variable-values-of-a-non-simplified-system-of-equations-from-a-structurally-simplified-solution-output/107915/14 "2024-01-11T17:46:00Z")
It hasn’t been optimized yet. We’re aware of this but first trying to get the whole interface together before doing more of these optimizations.