How should I simplify nested loops?

New to Julia
1

I am trying to understand the julia codes I am learning, and I find it difficult to understand the nested loop, don’t know how that should be simplified instead. Since I am comfortable with python, I tried to understand them in a pythonic way, but hard to difficult to understand the code, how those nested loops could be written explicitly or simplified. Can anyone point me out how a nested loop should be simlified? If it is possible, how should we write those down in pythonic way? any idea?

codes with nested loop

here is the function body of calc_dot in this gist

const_added = Tuple{Integer, vector}[]
const_prev = Tuple{Integer, vector}[]
n_added = length(const_added)
n_prev = length(const_prev )

the code blocks below are written in a compact way where a nested loop used. How should we simplify those?

Q_aug = [ ( calc_dot((const_prev[i][1], const_prev[i][2], const_added[j][1], const_added[j][2]), K, y) )::Float64
      for i=1:n_prev, j=1:n_added]

Q_aug_diag = [
      ( i >= j ? calc_dot((const_added[i][1], const_added[i][2], const_added[j][1], const_added[j][2]), K, y) : 0.0 )::Float64
      for i=1:n_added, j=1:n_added]

Q_aug_diag = [ (i >= j ? Q_aug_diag[i,j] : Q_aug_diag[j,i] )::Float64 for i=1:n_added, j=1:n_added ]

can anyone have a better idea above codes can be written in a simplistic way? Is that possible to write those in python? Thanks in advance!

1 Like
2

Loops are not to be avoided in Julia. They can be fast.

4 Likes
4

It might be helpful to post MWE.

5

right, it is nice to make MWE but I am new to this community, don’t have much experience in julialang. Honestly I do not know what’s best way to make MWE for this case. I am trying to learn julialang from forked github project, and above code blocks gave me hard to understand. Since data type are defined above, could you suggest or make solution for how should we write the above code in a simplistic way, especially how nested loop parts can be simplified? Thanks

6

Sorry, I don’t know what it is you’re trying to do.

8

To motivate what @PetrKryslUCSD is saying, I am some guy who happened to see this post and take a look. I figured I could probably help, so I copy-pasted your code into a file to play with it. But you can’t run those commands and get checkable output, so it’s hard to provide input. If you give us a script that runs so we can make sure that what we produce is the same output but with cleaner code, then it is much easier to help. As it stands, I kind of have to guess about what’s going on.

For example, I see you’re indexing const_prev, but it’s an empty collection from your original code. It’s hard to guess what’s happening in between the snippets you’ve provided.

Sorry, several edits. I wrote things too quickly.

1 Like
9

Oh, do you mean how to write the comprehensions in another way?
https://docs.julialang.org/en/v1/manual/arrays/#man-comprehensions

1 Like
10

Comprehensions are considered to be pythonic, so in that sense I think that part of the code is as it should be. If you want it simplified you probably must move away from pythonic and instead try something Julian like vectorizing by broadcasting. That might simplify the code.

But as some people have said now, your example code is incomplete, so we cannot understand it. For example, calc_dot seems to be missing a method, where’s the definition for the call occurring inside calc_dot?

11

currently input could be array or sparse matrix, or any other input type could be option. but the point is, instead of from specific input to get specific output which I don’t have any, I am trying to understand how those code works, especially nested loops part Q_aug, Q_aug_diag. I am experimenting those codes for using zero-sum game to solve classification tasks. Image, we have nxm data matrix, and d dim output label. I just added full function body of calc_dot in this gist

If those code can be simplified for any type of input, that would help me understand julialang convention. how should we write up nested loop logic in above code in simplistic way? Can we write code blocks above in python? Thanks!

12

I am sorry, I just missed that one. Here is whole function body of calc_dot: example function in gist

13

These are not nested loops, but rather matrix comprehensions. I suggest the opposite approach to what you suggested: turn those into actual nested loops of the form

Q = # allocate matrix
for i in ...
    for j in ...
        Q[i, j] = # calculation...
    end
end

This will make the path to simplification (if one exists) much clearer.

5 Likes
14

They would look basically the same in Python.

15

Thanks. How can we write those in simplistic way either in julialang or python? here is the full function body of calc_dot in this gist

Q_aug = [ ( calc_dot((const_prev[i][1], const_prev[i][2], const_added[j][1], const_added[j][2]), K, y) )::Float64
      for i=1:n_prev, j=1:n_added]

Q_aug_diag = [
      ( i >= j ? calc_dot((const_added[i][1], const_added[i][2], const_added[j][1], const_added[j][2]), K, y) : 0.0 )::Float64
      for i=1:n_added, j=1:n_added]

Q_aug_diag = [ (i >= j ? Q_aug_diag[i,j] : Q_aug_diag[j,i] )::Float64 for i=1:n_added, j=1:n_added ]

Could you elaborate on your thoughts? Thanks!

16

thanks. How should I understand this:

Q_aug_diag = [ (i >= j ? Q_aug_diag[i,j] : Q_aug_diag[j,i] )::Float64 for i=1:n_added, j=1:n_added ]

especially, what is the meaning of this: Q_aug_diag[i,j] : Q_aug_diag[j,i] )? is that like swap operation as Q_aug_diag[i,j] = Q_aug_diag[j,i] ) in pythonic way? any idea?

17 
condition ? if_true : if_false

is a ternary expression. You can rewrite that comprehension as the following loops

Q_aug_diag = zeros(n_added, n_added)

for j in 1:n_added
    for i in 1:n_added
        if i >= j
            Q_aug_diag[i, j] =  Q_aug_diag[i, j]
        else
            Q_aug_diag[i, j] = Q_aug_diag[j, i]
        end
    end
end

From which it is immediately clear that this is a very silly way of making Q_aug_diag symmetric.

On another note, calc_dot is quite poorly written (I’m sorry to say, it looks like a nightmare from matlab). It could be a good exercise in learning julia to simplify it as far as possible.

I also suggest going through the julia manual and looking through common julia packages to familiarize yourself further.

3 Likes
18

thanks for the thoughtful comment. I agree with your opinion and I am quite curious about julialang, especially if I am pursuing vectorized code for faster speed, julialang is pretty good I think.

Can I ask you one more question before accepting your solution as an answer? I am not understanding this code line below which is based on this thread:

    Q = [
          ( (i <= n_prev && j <= n_prev) ? Q_prev[i, j] : ( i <= n_prev ? Q_aug[i, j-n_prev] :
          ( j <= n_prev ? Q_aug[j, i-n_prev] : Q_aug_diag[i-n_prev, j-n_prev] ) ) )::Float64
          for i=1:n_const, j=1:n_const]

so I could understand how to rewrite nested loop by taking hints from your answer, but now this Q is really weird to me, looks there are multiple conditions in nested loop, which I hard to simplify them in julialang. Could you help me how this should be written in better way? Thanks a lot!

19

Yes that is indeed more complicated. In what way is it based on this thread? What is it for and can you simply delete it and start from scratch :stuck_out_tongue_winking_eye: ?

I would go through and rewrite it as a loop, then turn each ternary expression into the appropriate if-else statement. A chained ternary expression becomes a nested if-statement:

cond1 ? a : cond2 ? b : c

becomes

if cond1
    a
else
    if cond2
        b
    else
        c
    end
end

I would rather not go through your example myself, but hopefully that helps…

1 Like
21

I must be misunderstanding something, that gist does not contain a Q, nor any comprehensions or ternaries. I also do not see it in your original post. Regardless, I would go through the process I said: rewrite the comprehensions as loops, then rewrite the ternaries as if-else.

22

right, nothing to do with gist. I tried if-eles skeleton you showed, but I think I still get stuck with them. Can you show me how we could rewrite this Q in better way? Thanks a lot!

    Q = [
          ( (i <= n_prev && j <= n_prev) ? Q_prev[i, j] : ( i <= n_prev ? Q_aug[i, j-n_prev] :
          ( j <= n_prev ? Q_aug[j, i-n_prev] : Q_aug_diag[i-n_prev, j-n_prev] ) ) )::Float64
          for i=1:n_const, j=1:n_const]

espetially, this part is complicated to understand: Q_prev[i, j] : ( i <= n_prev ? Q_aug[i, j-n_prev] : ( j <= n_prev ? Q_aug[j, i-n_prev] : Q_aug_diag[i-n_prev, j-n_prev] ) ), I don’t know how should simplify entire things that assigned to Q as simple as possible? Any idea?

for instance, this is my pythonic way of writing Q that written in julialang above:

        Q = []
        for i in range(1, n_const):
            for j in range(1, n_const):
                if i< n_prev and j <= n_prev:
                        .....
                        .....

but I don’t know how to write the rest (complicated nested condition in matrix Q). Could you suggest any other way to write Q (easy to understand, and simple)? Thanks!

23

Take (condition ? thenthis : otherwisethat) and turn it into

if condition 
    thenthis 
else 
    otherwisethat
end

Is that what your question was about?

1 Like