StochasticPrograms.jl - DIfficulties with vector @uncertain inputs

Hi,

I’m testing the StochasticPrograms.jl package (Awesome work by the way!) on a simple renewables energy sizing use case. I have difficulty figuring out how to use uncertain parameters in the 2nd stage with array inputs.
All the examples I found declare single numbers.

What is the right way to declare uncertain parameters that are indexed both with a symbol and time index?

Here is where I’m at after reading parts of the documentation. It leads to an error but that’s probably that I’m not using the syntax right:

using StochasticPrograms, Cbc

horizon = 2
@scenario ResScenario = begin
    solar::Vector{Float64}
    wind_offshore::Vector{Float64}
    wind_onshore::Vector{Float64}

    @zero begin
        return ResScenario([0., 0.], [0., 0.], [0., 0.])
    end
end

res_model = @stochastic_model begin
    @stage 1 begin
        @parameters begin
            sources = [:solar, :wind_offshore, :wind_onshore]
            capex = Dict(
                :solar => 100,
                :wind_offshore => 150,
                :wind_onshore => 120)
        end
        @decision(model, capacity[s in sources] >= 0)
        @objective(model, Min,
            sum(capex[s] * capacity[s] for s in sources))
    end
    @stage 2 begin
        @parameters begin
            time = 1:horizon
            sources = [:solar, :wind_offshore, :wind_onshore]
            import_cost = 10_000
            demand = 10
        end
        @uncertain profile::ResScenario
        @variable(model, imports[t = time] >= 0)
        @variable(model, curtailed[t = time] >= 0)
        @objective(model, Min, sum(import_cost * imports[t] for t in time ))
        @expression(model, generation[t = time],
            sum(getproperty(profile, s)[t] * capacity[s] for s in sources))
        @constraint(model, supply_demand_balance[t = time],
            generation[t] + imports[t] - curtailed[t] == demand)
    end
end

ξ₁ = ResScenario([0.5, 0.5], [0.1, 0.1], [0.5, 0.5], probability = 1/3)
ξ₂ = ResScenario([0.1, 0.1], [0.3, 0.3], [0.5, 0.5], probability = 1/3)
ξ₃ = ResScenario([0., 0.], [0.7, 0.7], [0.3, 0.3], probability = 1/3)

res = instantiate(res_model, [ξ₁,ξ₂,ξ₃], optimizer = Cbc.Optimizer)

optimize!(res)

objective_value(res)

optimal_decision(res)

Thanks!

You didn’t post an error, but I’m guessing it’s something like:

ERROR: An object of name generation is already attached to this model. 
If this is intended, consider using the anonymous construction syntax, 
e.g., x = @variable(model, [1:N], ...) where the name of the object does 
not appear inside the macro.

If so, this is a problem with using @expression. I ran into a similar issue that was fixed (and I would’ve expected to resolve this one too).

Nevertheless, removing the expression and modifying your stage 2 constraint like so

@constraint(model, supply_demand_balance[t = time],
     sum(getproperty(profile, s)[t] * capacity[s] for s in sources) + imports[t] - curtailed[t] == demand)

seems to fix the problem in the short term.

Addressing your question about @uncertain inputs, what you did seems to be a fine solution to me.

Not that either of these are necessarily better, but for the sake of demonstration, I usually try to stick with the default Scenario unless I have some reason not to and take one of the following approaches:

  1. not as easy to read but usually easy to assemble via copy/paste
ξ₁ = Scenario(; (source => val for (source, val) in zip(sources, [[0.5, 0.5], [0.1, 0.1], [0.5, 0.5]]))..., probability = 1/3)
ξ₂ = Scenario(; (source => val for (source, val) in zip(sources, [[0.1, 0.1], [0.3, 0.3], [0.5, 0.5]]))..., probability = 1/3)
ξ₃ = Scenario(; (source => val for (source, val) in zip(sources, [[0.0, 0.0], [0.7, 0.7], [0.3, 0.3]]))..., probability = 1/3)

in this case you would have @uncertain profile[s in sources] and your constraint becomes

@constraint(model, supply_demand_balance[t = time],
     sum(profile[s][t] * capacity[s] for s in sources) + imports[t] - curtailed[t] == demand)

This works pretty well when I’ve already got the data in a conducive format, but I haven’t ever tried to extend it to uncertainties with more than 2 dimensions

  1. Using the @container_scenario macro which seems to be provided for this very kind of thing.
s₁ = Dict((:solar, 1) => 0.5, (:wind_offshore, 1) => 0.1, (:wind_onshore, 1) => 0.5,
          (:solar, 2) => 0.5, (:wind_offshore, 2) => 0.1, (:wind_onshore, 2) => 0.5)

s₂ = Dict((:solar, 1) => 0.1, (:wind_offshore, 1) => 0.3, (:wind_onshore, 1) => 0.5,
          (:solar, 2) => 0.1, (:wind_offshore, 2) => 0.3, (:wind_onshore, 2) => 0.5)

s₃ = Dict((:solar, 1) => 0.0, (:wind_offshore, 1) => 0.7, (:wind_onshore, 1) => 0.3,
          (:solar, 2) => 0.0, (:wind_offshore, 2) => 0.7, (:wind_onshore, 2) => 0.3)

ξ₁ = @container_scenario([s = sources, t = time], s₁[s,t], probability = 1/3)
ξ₂ = @container_scenario([s = sources, t = time], s₂[s,t], probability = 1/3)
ξ₃ = @container_scenario([s = sources, t = time], s₃[s,t], probability = 1/3)

in which case you would have @uncertain profile[s in sources, t in time] and the constraint is

@constraint(model, supply_demand_balance[t = time],
    sum(profile[s, t] * capacity[s] for s in sources) + imports[t] - curtailed[t] == demand)

(Note the slightly different indexing profile[s][t] vs profile[s, t] between the two)

2 Likes

Yes that’s the error I was having.
It works perfectly now, thank you! :100:

I will try the Scenario based syntax. I defined the specific type mainly because I did not figure out how to formulate it with the base tools. The examples you gave document this usage which is very helpful!

Hey,

I will look into the error when I find the time. Otherwise, I would be interested to hear if the container syntax works. This syntax is quite new, but it seems powerful enough that I might push harder for its usage. It should cover most common situations and for more complex cases @scenario can be kept as a fallback.

Note that the fallback expectation implementation for Scenario is currently only implemented for one-dimensional data. Adding it for JuMP containers should not be difficult, but until then you need to define it yourself if you want to use EVP, VSS, etc.

I hope that the package can be useful to you :slight_smile:

1 Like

Thank you @martinbiel
The package is really useful. I’m very interested by the alternative formulations you included (L-shaped & progressive hedging)

The @container_scenario worked following the steps explained by @rtwalker.
I tried to make a list comprehension to inject more scenarios. Passing a variable nb_scenarios to set the probability fails with the following error:

ERROR: LoadError: UndefVarError: nb_scenarios not defined

I’m still a rookie in Metaprogramming. Is there a special syntax to pass variables to a macro?

Could you include a complete snippet of the code that errors?

Your original @expression error should be fixed in master now. Do not hesitate to submit issues if you find more clear errors!

1 Like

Here is the full example (Do you prefer that I post it on Github?)

using StochasticPrograms, Cbc

horizon = 2

res_model = @stochastic_model begin
    @stage 1 begin
        @parameters begin
            sources = [:solar, :wind_offshore, :wind_onshore]
            capex = Dict(
                :solar => 100,
                :wind_offshore => 150,
                :wind_onshore => 120)
        end
        @decision(model, capacity[s in sources] >= 0)
        @objective(model, Min,
            sum(capex[s] * capacity[s] for s in sources))
    end
    @stage 2 begin
        @parameters begin
            time = 1:horizon
            sources = [:solar, :wind_offshore, :wind_onshore]
            import_cost = 10_000
            demand = [10, 8]
        end
        @uncertain profile[s in sources, t in time]
        @variable(model, imports[t = time] >= 0)
        @variable(model, curtailed[t = time] >= 0)
        @objective(model, Min, sum(import_cost * imports[t] for t in time ))
        @constraint(model, supply_demand_balance[t = time],
            sum(profile[s, t] * capacity[s] for s in sources)
            + imports[t] - curtailed[t]
            == demand[t])
    end
end

s₁ = Dict((:solar, 1) => 0.5, (:wind_offshore, 1) => 0.1, (:wind_onshore, 1) => 0.5,
          (:solar, 2) => 0.5, (:wind_offshore, 2) => 0.1, (:wind_onshore, 2) => 0.5)

s₂ = Dict((:solar, 1) => 0.1, (:wind_offshore, 1) => 0.3, (:wind_onshore, 1) => 0.5,
          (:solar, 2) => 0.1, (:wind_offshore, 2) => 0.3, (:wind_onshore, 2) => 0.5)

s₃ = Dict((:solar, 1) => 0.0, (:wind_offshore, 1) => 0.7, (:wind_onshore, 1) => 0.3,
          (:solar, 2) => 0.5, (:wind_offshore, 2) => 0.7, (:wind_onshore, 2) => 0.3)


nb_scenarios = 3
sources = [:solar, :wind_offshore, :wind_onshore]
ξ₁ = @container_scenario([s = sources, t = 1:horizon], s₁[s,t], probability = 1/nb_scenarios)
ξ₂ = @container_scenario([s = sources, t = 1:horizon], s₂[s,t], probability = 1/nb_scenarios)
ξ₃ = @container_scenario([s = sources, t = 1:horizon], s₃[s,t], probability = 1/nb_scenarios)

res = instantiate(res_model, [ξ₁,ξ₂,ξ₃], optimizer = Cbc.Optimizer)

optimize!(res)

objective_value(res)

optimal_decision(res)

In general, bugs and feature requests should ideally be posted on Github, while general usage questions is more appropriate here on discourse :slight_smile:

This particular error was a bug in @container_scenario. It is fixed on master now!

Got it!
I’m still learning the details of Julia / JuMP syntaxe so it’s not clear for me to make the difference between bugs and my errors … but it’s getting better everyday :sweat_smile:

Thank you @martinbiel for your reactivity. Really nice!

If the error occurs when you run a function from the package you can probably assume that it is a bug :wink:

Hi,
I have near the same stochastic project as @elalaouifaris had. I tried the second way to create the scenarios, but I faced to this error:
LoadError: UndefVarError: @container_scenario not defined

I imported these packages:
using StochasticPrograms, Cbc

I would be grateful if you or other guys (@elalaouifaris and @martinbiel ) have a solution.

Thanks!