Starting to learn about optimization

Specific Domains Optimization (Mathematical)
1

I’m considering tackling a problem that appears to have a fixed handful of cutoffs and a performance metric, so I started reading a bit about optimization with essentially zero initial knowledge (not sure ordinary least squares counts for much). I’m reading about many tools, but I keep running into what I think to be important practical considerations that aren’t answered in one place. I’m not writing anything yet, but it could be helpful if I don’t have these questions hanging over my head:

  • These if-else cutoffs are not differentiable, to my understanding. They are not obvious step functions either, rather decisions in a loop. Does this mean smoothing approximations are not possible and I need to focus on derivative-free optimization?
  • I’m seeing vague hints of downsides to derivative-free optimization like poor scaling, slow convergence, and more likely non-global optima. What are some practical things I should know?
  • What do I do when out-of-sample testing suggests the model is bad? I’d just get the same result if I try again without changing anything.

If this doesn’t concern Julia packages much, I’ll move it to Off-Topic.

2

tackling a problem that appears to have a fixed handful of cutoffs and a performance metric,

What is the specific problem? I assume the “fixed handful of cutoffs” are constraints? Performance metric is probably the objective function. If you’re solving optimizations with constraints, then probably you need to use JuMP-like workflow (you need an underlying solver).

These if-else cutoffs are not differentiable

Maybe you can present the specific problem. To me it sounds like integer programming, e.g. in a unit commitment problem if the generator is ON, then it’s output power is e.g. [2MW, 10MW], if it’s OFF, then it’s output power is 0MW.

rather decisions in a loop

I didn’t get this either. What does loop here mean? Do you mean reinforcement learning (aka approximate dynamic programming)?

Does this mean smoothing approximations are not possible and I need to focus on derivative-free optimization?

No. There’re two layers, the outer layer is branch-and-bound, the inner layer is continuous optimization where you apply effective optimization methods. It’s not quite accurate to partition the methods into derivative-free methods or others. For example, the well-studied simplex algorithm for linear programming—it is not a typical derivative-driven algorithm, but it’s very effective.

downsides to derivative-free optimization like poor scaling, slow convergence, and more likely non-global optima.

You need to distinguish optimization problems and algorithms. The primary question is: is my optimization convex? If yes, then it is easy, otherwise pursuing global optimum is hard. If it’s a nonconvex differentiable nonlinear programming, then derivative-based methods are usable, which is based on descent direction and stepsize selection (But they may converge to local optimum, so they essentially don’t have any guarantee about the quality of the solution. Or, say, different local optimization algorithms are only relatively comparable). So the core issue is not about “using derivatives or not”.

What do I do when out-of-sample testing suggests the model is bad?

Are you doing stochastic optimization? In stochastic optimization, there are two models: the model of the uncertainty, and the model of physic system. For example, if you are solving a stochastic unit commitment, then you need a model for wind generation, maybe an ARIMA model for instance. This part is irrelevant to optimization techniques.

3

The description is fairly shaky at this point, but maybe it’d help narrow down to some class of problems. A programmable HVAC system has 3 visible states to regulate indoor air temperature: heating, cooling, off. These states can be triggered at specified times or indoor air temperature cutoffs for possibly limited periods per day. These cutoffs could change dynamically e.g. cooling to 75F is triggered at >80F for the first time of the day but at >76F for the next 8 hours of maintenance. The simplest setting is to maintain one ideal temperature all the time. However, this wastes energy because there are long periods each day where nobody is home, and occupants change their ideal temperature across different seasons and can tolerate short periods of wider temperature ranges. So, the objective is to find an acceptable program that saves as much energy as possible.

There’s a fair bit of work I’d have to do before I even begin an optimization problem. I don’t have a measure of energy usage, just a few rough ideas of estimation. To evaluate the program, I’d have to look for historical hourly air temperatures and weather (sunny days heat the home through radiation as well) and model how those affect indoor air temperatures independently of the HVAC system. That’s mainly why I asked the questions in such a generic way, but for the purposes of the topic, we can assume I already have those figured out or address a different problem involving decisions in reaction to a non-stationary stochastic process.

4

This is called Home Energy Management System (HEMS) or Demand Response (DR), according to my knowledge (I also have been working on this topic for about 1 year).

It is indeed studied as a typical multiperiod stochastic optimal control problem.

I can recommend two references that I myself had read many times.

[1] A nonconvex nonlinear optimization: Online Learning and Distributed Control for Residential Demand Response | IEEE Journals & Magazine | IEEE Xplore (look for Eq.(9)(10))

[2] A MILP optimization: A Decentralized Framework for the Optimal Coordination of Distributed Energy Resources | IEEE Journals & Magazine | IEEE Xplore (look for Eq.(12)(13))

5

It depends on what the goal is, but the internals of HVAC control are typically not part of the EMS itself. HVAC control requires deep knowledge about the specific hardware being used, which an EMS vendor doesn’t have access to. Source: worked at a heat pump manufacturer and this was basically my job…

You can model this as being non-convex and use general MINLP techniques, but it will be computationally very costly. These methods will not run on HVAC hardware so that excludes local control.

The alternative to keep the problem convex is to say that q = q_\text{heat} - q_\text{cool}. Now you can impose individual constraints, cost functions and efficiency curves on both heating and cooling.

This formulation has one obvious problem, which is that [q_{\min}, q_{\max}] with q_{\min} > 0 has a feasible set that’s a union of \{0\} with an interval. This will bite you in low-load conditions because it is non-convex. After trying to find ways around this for a long time I concluded that the most practical solution is to do what HVAC control is already doing anyway, use an energy integral. Basically, if your q is not feasible (too low), stay off for a proportion of time equal to T\left(\frac{q_{min}-q}{q_{min}}\right). Real systems use a T of about 1-2 hours, check https://heatpumpmonitor.org/. This works because thermal dynamics of a building are very slow.

For control purposes you can use a basic thermal model, for example this 1 mass RC model:

C\,\frac{dT_{\text{in}}}{dt} = \frac{T_{\text{out}}(t) - T_{\text{in}}(t)}{R} + q(t) + w(t)

This is easy to extend with more information if you have it available. I ended up with a 3 mass model, for the interior, walls and supply system. This last one is especially important for buildings with larger radiators or underfloor heating. You may skip it for AC units. Since it is a linear ODE model, all the nice control theory you may learn at university applies. And you can generate very efficient convex MPC controllers for this problem (for example, with ModelPredictiveControl.jl)

6

I’m not sure how useful this clarification is, but by “acceptable program” I meant manually punching in a daily schedule or temperature thresholds on a thermostat with a fixed program (I assume it periodically checks the temperature for potential changes, hence “decisions in a loop”), not that I can load arbitrary programs onto the HVAC system or hook a separate computer to control it. I might tolerate doing that once a week at most, then it’s crossing fingers it performs well enough. I believe I can set decent parameters already, but I was hoping I can validate or improve upon that initial guess with recent publicly available weather data. I only have a middling 5yo laptop to do any optimization though, and I’m not keen on occupying it for many hours at a time.

7

I was able to get good results with Metaheuristics.jl to optimize non-differentiable decision-making algorithms.

8

Interesting, that doesn’t look like it’s under the Optimization.jl umbrella. Do they do distinct things? I can’t tell because I don’t know anything.

9

Haven’t tried Optimization.jl, but its README seems to mention Metaheuristics.jl as one of the backends, and it has other backends with similar functionalities, too.

10

If the goal is to learn about optimization I wouldn’t start here. These methods are useful when you are really stuck, but they work on black-box problems, are incredibly slow compared to classic optimization, and provide no guarantees. I’ve personally never needed them.

11

In building our house, I focused more on the house part of the equation than the controls, so my comments focus on that part, not your specific question.

It is worth considering the principles of what is a comfortable environment. Indeed what do we mean when we talk about room temperature? There are at least 3 factors important for a comfortable environment, mean radiant temperature, air temperature, and relative humidity, and we should probably include air speed.

I suggest reading the Chapter “Physiological Principles for Comfort and Health” from the ASHRAE Handbook of Fundamentals.

Then if you want a really detailed model of your home, I suggest the open source software from University of Strathclyde, called ESP-r. It has a bit of a learning curve, but I suspect it is still the most advanced modeling tool. It also has weather files for many locations, and if I recall directions for obtaining more weather files for other locations. Windows are also extremely important. I used Windows software from the US government for center of glass thermal modeling, and then converted the output into an Energy Rating to be able to select the best glass for the house over the heating season. Europe has software similar to Windows.

It would be interesting to combine information from a weather forecast, time of day electricity charges, house comfort requirements and heat pump efficiency to optimize the use of the heating plant. I am guessing someone has already done this?

Since this post is about HVAC optimization, I am wondering if you can change the title to “HVAC optimization” of something similar.

12

There are complicated models available (PMV/PPD based), which are absolutely useful when constructing comfortable environments or buildings for people. But in my (practical) experience these models are rarely useful for control; they require too many parameters which we have no or limited knowledge of. Humidity, MRT, metabolic rate, clothing insulation, air speed all factor into ‘perceived comfort’, and even then you are dealing with diverse people with diverse preferences. Most humans just want the HVAC to adhere to the temperature setpoint.

There are many papers on this topic. I recommend reading work from Lieve Helsen (KU Leuven). Her research group has commercialized the research through https://builtwins.com/. I think the main challenge for applications in residential buildings is still on how to do robust system identification and obtain a model that is useful for control applications. The models are simple, but the data of real systems is always of such terrible quality, that simple least squares finds models that are totally useless for control