Sum() syntax

jump

#1

Hello, from the JuMP quick guide, i’ve seen i can use the sum() function with a condition, like this:

sum(expression for i = I1, j = I2 if cond)

is the same as:

a = zero(AffExpr)
for i = I1
    for j = I2
        ...
        if cond
            a += expression
        end
        ...
    end
end

Does it work if i have another condition inside that first condition? If it does, how can i write it? Using commas to separe the if's, just like the for's?

To be more clear, i have this loop in my code i wish to change using the sum() syntax:

hmax = 25;
Nt = 17;
Yh = zeros(Complex128,Nt,Nt,hmax);

for h = 3:2:hmax
  for L = 1:Nt
    for C = 1:Nt
      for i = 1:Nt
        if CONDITION 1
            if CONDITION 2
                 Yh[L,C,h] += first_expression;
            end
            if CONDITION 3
                Yh[L,C,h] += second_expression;
            end
        end
        if CONDITION 4
                 Yh[L,C,h] += third_expression;
                 Yh[C,L,h] = Yh[L,C,h];
        end
      end
    end
  end
end

I thought in using 3 sum() here:

1 - The first for the conditions 1 and 2;
2 - The second for the conditions 1 and 3;
3 - The third for the condition 4.


#2

Might want to post this in the optimization category instead (https://discourse.julialang.org/c/domain/opt) since this is not about general Julia usage.


#3

Moved :slight_smile:


#4

The first thing to know is that sum (and the generator syntax inside it) is just standard Julia syntax. Since Yh contains Complex128, this doesn’t actually touch any code in JuMP.

The second thing is, don’t be afraid of loops. They might actually be easier to read than some complicated sequence of sum's. For example:

for h in 3:2:hmax, L in 1:Nt, C in 1:Nt, i in 1:Nt
    if condition_1 && condition_2
        Yh[L, C, h] += first_expression
    elseif condition_1 && condition_3
        Yh[L, C, h] += second_expression
    elseif condition_4
        Y[L, C, h] += third_expression
        Y[C, L, h] = Y[L, C, h]
    end
end

#5

I’m not sure where it’s documented (I’m certain it was at some point), but that if is a part of comprehension generator syntax, not the sum function.

EDIT: Nevermind, I found it - under Generator Expressions at the very bottom of the section.


#6

I’m sorry, my fault, that was wrong, it’s not a Complex128 variable anymore. Its just a real matrix.

Yh = zeros(Nt,Nt,hmax);

I’m asking about this sum() syntax, because i want to use the macros @constraint and/or @NLconstraint using those first, second and third expressions. I’m a bit confused in how to do this, but in my head it would be something like:

@constraint(m, Yh[L,C,h] ==  sum(first_expression for h in 3:2:hmax, L in 1:Nt, C in 1:Nt, i in 1:Nt if condition_1 && condition_2))

@constraint(m, Yh[L,C,h] == sum(second_expression for h in 3:2:hmax, L in 1:Nt, C in 1:Nt, i in 1:Nt if condition_1 && condition_3))

@constraint(m, Yh[L,C,h] == sum(third_expression for h in 3:2:hmax, L in 1:Nt, C in 1:Nt, i in 1:Nt if condition_4))

And not yet sure how i would write the Yh[C,L,h] = Yh[L,C,h] part.


#7

You need something like

for h in 3:2:hmax, L in 1:Nt, C in 1:Nt
    @constraint(m, Yh[L, C, h] == sum(first_expression for i in 1:Nt if condition_1 && condition_2))
    if condition_4
        @constraint(m, Yh[C, L, h] == Yh[L, C, h])
    end
end

#8

Hi, so i have the following:

i = 1; # Had to add this, otherwise it produces an error saying i is not defined
for h in 3:2:hmax, L in 1:Nt, C in 1:Nt
    if (condition_1 && condition_2)
       @NLconstraint(m, Yh[L, C, h] == sum(first_expression for i in 1:Nt))
    elseif (condition_1 && condition_3)
       @constraint(m, Yh[L, C, h] == sum(second_expression for i in 1:Nt))
    elseif (condition_4)
       @constraint(m, Yh[L, C, h] == sum(third_expression for i in 1:Nt))
       @constraint(m, Yh[C, L, h] == Yh[L, C, h])
    end
end

But when i run my code, i get zeros in all positions, except for:

julia> getvalue(Yh)
Yh: 3 dimensions:
[ 1,:,:]
  [ 1, 1,:]
    [ 1, 1, 3] = -46.731813188856584
    [ 1, 1, 5] = -28.391636812292383
    [ 1, 1, 7] = -20.2401694582778
    [ 1, 1, 9] = -15.63001153228338
    [ 1, 1,11] = -12.653536842073537
    [ 1, 1,13] = -10.562937348884866
    [ 1, 1,15] = -9.00594278820859
    [ 1, 1,17] = -7.7950523228143975
    [ 1, 1,19] = -6.821345536276918
    [ 1, 1,21] = -6.017242413033545
    [ 1, 1,23] = -5.338598503936174
    [ 1, 1,25] = -4.755360420338826

Every other single position is zero, like this, for example:

[ 8, 6,:]
    [ 8, 6, 3] = 0.0
    [ 8, 6, 5] = 0.0
    [ 8, 6, 7] = 0.0
    [ 8, 6, 9] = 0.0
    [ 8, 6,11] = 0.0
    [ 8, 6,13] = 0.0
    [ 8, 6,15] = 0.0
    [ 8, 6,17] = 0.0
    [ 8, 6,19] = 0.0
    [ 8, 6,21] = 0.0
    [ 8, 6,23] = 0.0
    [ 8, 6,25] = 0.0

Any ideas what could be causing this error? I’ve checked all the vectors in those first, second and third_expressions, and they all got real values inside.

It is because i took the if conditions out of the sum() argument? They really need to be inside it? Or there is something else causing this?


#9

If your conditions depend on i, they must be inside of the sum expression.


#10

Yeah, my conditions depends on i, L and C.

So, back with the conditions inside the sum(), i got this now:

i = 1; # Had to add this, otherwise it produces an error saying i is not defined
for h in 3:2:hmax, L in 1:Nt, C in 1:Nt
    @constraint(m, Yh[L, C, h] == sum(first_expression for i in 1:Nt if condition_1 && condition_2))
    @constraint(m, Yh[L, C, h] == sum(second_expression for i in 1:Nt if condition_1 && condition_3))
    @constraint(m, Yh[L, C, h] == sum(third_expression for i in 1:Nt if condition_4))
    if condition_4
    @constraint(m, Yh[C, L, h] == Yh[L, C, h])
    end
end

But i’m getting this error:

ERROR: LoadError: AssertionError: line[1:7] == "Options"
Stacktrace:
 [1] read_sol(::IOStream, ::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:666
 [2] read_results(::IOStream, ::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:575
 [3] open(::AmplNLWriter.##117#118{AmplNLWriter.AmplNLMathProgModel}, ::String, ::String) at .\iostream.jl:152
 [4] read_results(::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:569
 [5] optimize!(::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:419
 [6] optimize!(::CouenneNL.CouenneNLNonlinearModel) at C:\Users\Muril\.julia\v0.6\Lazy\src\macros.jl:268
 [7] #solvenlp#165(::Bool, ::Function, ::JuMP.Model, ::JuMP.ProblemTraits) at C:\Users\Muril\.julia\v0.6\JuMP\src\nlp.jl:1271
 [8] (::JuMP.#kw##solvenlp)(::Array{Any,1}, ::JuMP.#solvenlp, ::JuMP.Model, ::JuMP.ProblemTraits) at .\<missing>:0
 [9] #solve#116(::Bool, ::Bool, ::Bool, ::Array{Any,1}, ::Function, ::JuMP.Model) at C:\Users\Muril\.julia\v0.6\JuMP\src\solvers.jl:172
 [10] solve(::JuMP.Model) at C:\Users\Muril\.julia\v0.6\JuMP\src\solvers.jl:150
 [11] include_string(::String, ::String) at .\loading.jl:522
 [12] include_string(::Module, ::String, ::String) at C:\Users\Muril\.julia\v0.6\Compat\src\Compat.jl:88
 [13] (::Atom.##112#116{String,String})() at C:\Users\Muril\.julia\v0.6\Atom\src\eval.jl:109
 [14] withpath(::Atom.##112#116{String,String}, ::String) at C:\Users\Muril\.julia\v0.6\CodeTools\src\utils.jl:30
 [15] withpath(::Function, ::String) at C:\Users\Muril\.julia\v0.6\Atom\src\eval.jl:38
 [16] hideprompt(::Atom.##111#115{String,String}) at C:\Users\Muril\.julia\v0.6\Atom\src\repl.jl:67
 [17] macro expansion at C:\Users\Muril\.julia\v0.6\Atom\src\eval.jl:106 [inlined]
 [18] (::Atom.##110#114{Dict{String,Any}})() at .\task.jl:80
while loading C:\Users\Muril\Desktop\Murilo\UFPE\Dissertação\Julia\Arquivos Feitos\teste_otimização_formação_Yh.jl, in expression starting on line 198

Wich i have no idea what it means. I remember i already got it somewhere else.


#11

This is a bug in AmplNLWriter. It will be fixed in an upcoming release. See https://github.com/JuliaOpt/AmplNLWriter.jl/pull/81 for details.


#12

Oh, thanks for the reply! Well, i’ll have to wait then… hope it get fixed anytime soon!


#13

Please follow the instructions in https://github.com/JuliaOpt/AmplNLWriter.jl/issues/80#issuecomment-421447842 to see if it fixes your problem.


#14

I don’t know if i did correctly, but this is what i got:

julia> Pkg.checkout("AmplNLWriter")
INFO: Checking out AmplNLWriter master...
INFO: Pulling AmplNLWriter latest master...
INFO: No packages to install, update or remove
julia> Pkg.free("AmplNLWriter")
INFO: Freeing AmplNLWriter
INFO: No packages to install, update or remove

And the error persists.


#15

Just run Pkg.checkout("AmplNLWriter") until https://github.com/JuliaLang/METADATA.jl/pull/18026 is merged. Then, once it is merged, run Pkg.free("AmplNLWriter") and then Pkg.update().


#16

I tried again typing those commands in the Julia command prompt, and then the Pkg.update() one. It updated a lot of things, but the error still occurs.

After the Pkg.update(), i restarted Atom, and when i ran my code, these messages appeared.

INFO: Recompiling stale cache file C:\Users\Muril\.julia\lib\v0.6\DiffResults.ji for module DiffResults.
INFO: Recompiling stale cache file C:\Users\Muril\.julia\lib\v0.6\JuMP.ji for module JuMP.
INFO: Recompiling stale cache file C:\Users\Muril\.julia\lib\v0.6\CouenneNL.ji for module CouenneNL.

That was the only different thing that happened. After that, the error message appears again.

Couenne 0.5.6 -- an Open-Source solver for Mixed Integer Nonlinear Optimization
Mailing list: couenne@list.coin-or.org
Instructions: http://www.coin-or.org/Couenne
couenne:
ANALYSIS TEST: Reformulating problem: 2.3 seconds
Loaded instance "C:\Users\Muril\.julia\v0.6\AmplNLWriter\.solverdata\jl_94F4.tmp.nl"
Constraints:        21352
Variables:           8160 (51 integer)
Auxiliaries:         1922 (17 integer)

Problem infeasible
Total solve time:                           0.001s (0.001s in branch-and-bound)
Lower bound:                                 -inf
Upper bound:                                  inf  (gap: --)
Branch-and-bound nodes:                         0
ERROR: LoadError: AssertionError: line[1:7] == "Options"
Stacktrace:
 [1] read_sol(::IOStream, ::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:666
 [2] read_results(::IOStream, ::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:575
 [3] open(::AmplNLWriter.##117#118{AmplNLWriter.AmplNLMathProgModel}, ::String, ::String) at .\iostream.jl:152
 [4] read_results(::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:569
 [5] optimize!(::AmplNLWriter.AmplNLMathProgModel) at C:\Users\Muril\.julia\v0.6\AmplNLWriter\src\AmplNLWriter.jl:419
 [6] optimize!(::CouenneNL.CouenneNLNonlinearModel) at C:\Users\Muril\.julia\v0.6\Lazy\src\macros.jl:268
 [7] #solvenlp#165(::Bool, ::Function, ::JuMP.Model, ::JuMP.ProblemTraits) at C:\Users\Muril\.julia\v0.6\JuMP\src\nlp.jl:1271
 [8] (::JuMP.#kw##solvenlp)(::Array{Any,1}, ::JuMP.#solvenlp, ::JuMP.Model, ::JuMP.ProblemTraits) at .\<missing>:0
 [9] #solve#116(::Bool, ::Bool, ::Bool, ::Array{Any,1}, ::Function, ::JuMP.Model) at C:\Users\Muril\.julia\v0.6\JuMP\src\solvers.jl:172
 [10] solve(::JuMP.Model) at C:\Users\Muril\.julia\v0.6\JuMP\src\solvers.jl:150
 [11] include_string(::String, ::String) at .\loading.jl:522
 [12] include_string(::Module, ::String, ::String) at C:\Users\Muril\.julia\v0.6\Compat\src\Compat.jl:88
 [13] (::Atom.##112#116{String,String})() at C:\Users\Muril\.julia\v0.6\Atom\src\eval.jl:109
 [14] withpath(::Atom.##112#116{String,String}, ::String) at C:\Users\Muril\.julia\v0.6\CodeTools\src\utils.jl:30
 [15] withpath(::Function, ::String) at C:\Users\Muril\.julia\v0.6\Atom\src\eval.jl:38
 [16] hideprompt(::Atom.##111#115{String,String}) at C:\Users\Muril\.julia\v0.6\Atom\src\repl.jl:67
 [17] macro expansion at C:\Users\Muril\.julia\v0.6\Atom\src\eval.jl:106 [inlined]
 [18] (::Atom.##110#114{Dict{String,Any}})() at .\task.jl:80
while loading C:\Users\Muril\Desktop\Murilo\UFPE\Dissertação\Julia\Arquivos Feitos\teste_otimização_formação_Yh.jl, in expression starting on line 283

Sorry, i don’t understand much of these things… but what do you mean with:

Just run Pkg.checkout("AmplNLWriter") until https://github.com/JuliaLang/METADATA.jl/pull/18026 is merged

I just ran that command, and waited until it was done, so i could ran Pkg.free("AmplNLWriter"). Then i waited again, until i could Pkg.update().
Is this correct? Or i did something wrong?


#17

Here is a step-by-step guide:

Step 1. Run Pkg.checkout("AmplNLWriter")
Step 2. Test your code. Hopefully it works. If it doesn’t please follow the instructions in this Github issue: https://github.com/JuliaOpt/AmplNLWriter.jl/issues/80#issuecomment-421127894
Step 3. Wait a few days. You can keep working on your (hopefully working) code in the meantime.
Step 4. Check https://github.com/JuliaLang/METADATA.jl/pull/18026 to see if the green “Open” symbol has turned to a purple “Merged” symbol. If it hasn’t, go back to working on your code and wait a few more days. If it has, go to step 5.
Step 5. Run Pkg.free("AmplNLWriter")
Step 6. Run Pkg.update()
Step 7. Test your code again. Everything should be fixed.


#18

Just tried again, and i got this:

julia> Pkg.checkout("AmplNLWriter")
INFO: Checking out AmplNLWriter master...
INFO: Pulling AmplNLWriter latest master...
WARNING: julia is fixed at 0.6.3 conflicting with requirement for Makie: [0.7.0,∞)
WARNING: julia is fixed at 0.6.3 conflicting with requirement for AbstractPlotting: [0.7.0,∞)
ERROR: fixed packages introduce conflicting requirements for GLFW:
         Makie requires versions [2.2.0,∞) [none of the available versions can satisfy this requirement]
       available versions are 0.0.0, 0.0.1, 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-alpha.2, 1.0.0-alpha.3, 1.0.0-alpha.4, 1.0.0-alpha.5, 1.0.0-alpha.6, 1.0.0-alpha.7, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.4.0, 1.4.1, 1.5.0, 2.0.0-alpha and 2.0.1

And the code is not working yet. Guess i’ll wait a few days then.

Thanks again for your assistance!


#19

Do you need Makie? Does it even work? If not, run Pkg.rm("Makie"). This error isn’t related to AmplNLWriter.


#20

Actually i don’t. It used to work, but i’m not going to use it anymore.

I still see the green “Open” symbol there, so i’ll wait a few more days.