# How to generate a customized \`for\` loop code?

**URL:** <https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552>\
**Category:** General Usage\
**Tags:** question, macros, metaprogramming\
**Created:** [November 21, 2023, 11:22pm UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552 "2023-11-21T23:22:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![frankwswang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/frankwswang/32/18561_2.png) [@frankwswang](https://discourse.julialang.org/u/frankwswang)\
**Post date:** [November 21, 2023, 11:22pm UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/1 "2023-11-21T23:22:11Z")

</div>

I’m using the macro `@hyperopt` from package [hyperopt.jl](https://github.com/baggepinnen/Hyperopt.jl) to optimize some hyperparameters for a machine learning model. However, I’m pretty constrained by the syntax of `@hyperopt`, which is designed to be used as follows:

```julia
hoRes = @hyperopt for i=numOfSamples, sampler=someSampler,
    hPar1 = valRange1, 
    hPar2 = valRange2,
    ...
    hParN = valRangeN
    
    modelFunc([hPar1, ..., hParN])
end

```

I want to wrap the above code in a function that can take a non-fixed number (`N`) of hyperparameters as a `Vector` argument as I’m testing many different variants of my model. However, due to the macro formalism, I cannot find a straightforward way to achieve this without explicitly writing out the `N` functions, each corresponding to optimizing a specific number (1 \<= I \<= `N`) of hyperparameters.

I believe there is a way to write a generated function (or a macro), `genFunc`, that directly modifies the code for the `for` loop when I specify the number of hyperparameters as input. It probably should look like something as below:

```julia
function wrapperOpt(::Val{N}, pars, valRanges, numOfSamples, someSampler) where {N}
    # N == length(pars) == length(valPranges)
    f = genFunc(Val(N), modelFunc, numOfSamples, someSampler)
    f(pars, valRanges)
end

```

where `genFunc` modifies the code involving `@hyperopt` and the required `for` loop.

However, there seems not to be an example of how to generate a code of for loop in the official documentation (especially with multiple condition expressions). And I don’t have much experience with metaprogramming either. If someone could help me out, I would much appreciate it! Thank you!

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [November 22, 2023, 7:01am UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/2 "2023-11-22T07:01:00Z")

</div>

Macros modify expressions when parsed from the text, before they’re evaluated. Since you’re passing in an evaluated type parameter with a value, macros work too soon. With much more limitations, `@generated` functions work on input types and generate the function body expression. That is plausible. It won’t work like your `genFunc`, a `@generated` function _is_ the callable; it makes its own code body expression, not another callable. So you’d want to pass in `pars` and `valRanges` so the `@hyperopt` loop expression has something to reference. That’s about as far as I can surmise because without providing specific examples of `pars, valRanges, numOfSamples, someSample`, no one else can begin on crafting the expression. It’s also not certain from HyperOpt.jl’s documentation whether it’s possible to do something like `hPar2 = valRanges[2]` accessing `valRanges` from outside the loop, all the examples seem to create their ranges inside the loop. Hopefully it is, because it’s in maintenance mode and can’t change in the foreseeable future.

Some quick tips on working on expressions since you’re doing the work:

1. To access source code as an expression, paste it in `x = quote ... end` to make an `Expr` instance; the assignment is just to keep it live. Don’t worry about pasting macro calls, they’re not expanded yet at the early phase of parsing where `Expr` are made.
2. Normally when an `Expr` is printed, it’ll try to show its source code appearance. To see its more direct abstract syntax tree (AST) structure, use `dump(x)`. Note that `dump` has to enter the tree recursively and has a default depth of 8; if that’s not enough to reveal some important parts of the tree (you’ll see `Expr` nodes), change it `dump(x; maxdepth=12)`.
3. Push comes to shove, you can insert code (`Symbol` and `Expr`) into an `Expr` by `push!`ing to some branch of the tree because it’s built out of `Vector{Any}`. When you can however, work by `$`-interpolating into a `quote end` block that looks more like the source code, it’s much easier to read and edit.

---

<div class="post-metadata">

**Author:** ![frankwswang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/frankwswang/32/18561_2.png) [@frankwswang](https://discourse.julialang.org/u/frankwswang)\
**Post date:** [November 22, 2023, 7:21pm UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/3 "2023-11-22T19:21:30Z")

</div>

Thanks for the reply!

I forgot that the `@generated` function does not support closure, so we can’t write an anonymous function inside an `Expr` to be generated when it is first called with a specific `Val(N)`.

However, I don’t understand why we cannot design a macro to automate the process of generating functions like the following without knowing the types of `x` and `y`:

```julia
function f2(x, y)
    @hyperopt for i=1:2, sampler=y, 
        x1 = x[1],
        x2 = x[2]
        model([x1, x2])
    end
end

function f3(x, y)
    @hyperopt for i=1:2, sampler=y, 
        x1 = x[1],
        x2 = x[2],
        x3 = x[3]
        model([x1, x2, x3])
    end
end

```

As far as I understand, we just need to know how to construct a `for` loop expression without explicitly writing it out but using some functions to build the sub-expressions and concatenate them. I found a possibly relevant [thread](https://discourse.julialang.org/t/how-does-julia-merge-several-exprs-to-one-expr/77498/6) about concatenating assignment expressions, but I don’t know how to make them part of the `for` loop expression. I think if I use the `dump` function you mentioned to analyze the expression structure of a for loop, then I might be able to do it.

If you have better ideas, I would much appreciate it! FYI, an example of those variables are:

```julia
varRanges = LinRange.([0:0.01:1, 0:0.02:2, 0:0.03:6]) # the values (ranges) for all the parameters
numOfSamples = 10 # to be assigned to `i` in the for loop
someSample = Hyperopt.RandomSampler() # The sampler provided by Hyperopt.jl

```

We actually don’t need `pars` if we can name the hyperparameters inside the macro before for loop expression. e.g., construct `pars = [Symbol(:a, j) for j=1:N]`, then interpolate its element values as the variable names for the assignments in the expression.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [November 22, 2023, 7:43pm UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/4 "2023-11-22T19:43:19Z")

</div>

Yep, can’t make functions in generated function bodies.

> [@frankwswang](#):
>
> However, I don’t understand why we cannot design a macro to automate the process of generating functions like the following without knowing the types of `x` and `y`

The types of `x` and `y` do appear irrelevant in this case. It’s mostly the `Val(N)` part that can’t have a value in the phase when a macro works, and you really need that `N` value to craft the method body. A normal function however can have a value for `Val(N)` at runtime and use it to craft an expression, which you can `eval` in the global scope. Maybe that’s feasible too.

Thank you for the examples, so let me ask this in return to check if it’s feasible for `@hyperopt`’s transformed expression: does this code run properly? If you paste this into a trivial `@generated` function with no processing, does the function run properly without hitting some `@generated` limitations?

```julia
# this would be what genFunc does
# hopefully as arguments
varRanges = LinRange.([0:0.01:1, 0:0.02:2, 0:0.03:6])
# these two could be baked in via Val
numOfSamples = 10
someSample = Hyperopt.RandomSampler()

# quote
  @hyperopt for i=1:numOfSamples, sampler=someSample,
    a1 = varRanges[1],
    a2 = varRanges[2],
    a3 = varRanges[3]
      model([a1, a2, a3])
  end

```

PS format those function examples with these

````julia
```
code here
```
````

---

<div class="post-metadata">

**Author:** ![frankwswang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/frankwswang/32/18561_2.png) [@frankwswang](https://discourse.julialang.org/u/frankwswang)\
**Post date:** [November 23, 2023, 6:40am UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/5 "2023-11-23T06:40:07Z")

</div>

Did you mean something like this?

```julia
model(var) = sum(@. var[1] + (var[2]-3)^2+ (var[3]-100)^2)

@generated function f()
    varRanges = LinRange.([0:0.01:1, 0:0.02:2, 0:0.03:6])
    numOfSamples = 10
    someSample = Hyperopt.RandomSampler()
    quote
        @hyperopt for i=1:numOfSamples, sampler=someSample,
            a1 = varRanges[1],
            a2 = varRanges[2],
            a3 = varRanges[3]
            model([a1, a2, a3])
        end
    end
end

```

Unfortunately, it did not work:

```julia
ERROR: The function body AST defined by this @generated function is not pure. This likely means it contains a closure, a comprehension or a generator.

```

P.S. Thanks for the reminder! I knew the usage of ````` but forgot.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [November 23, 2023, 7:22am UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/6 "2023-11-23T07:22:00Z")

</div>

Shame, nothing else in the returned `Expr` has those, so it’s `@hyperopt` doing something.

We’ve gone over how we can’t have the necessary information at parse-time (and your input isn’t source code anyway), and now that we know `@hyperopt` does things that compile-time `@generated` functions can’t do, the remaining option is a runtime `eval` of an expression returned by a function. Since runtime is the latest phase, there’s a lot of things that can’t be changed, but at the same time we have more information.

```julia
function genFuncExpr(N)
  pars = [Symbol(:a, j) for j=1:N]
  funcexpr = quote
    function (varRanges, numOfSamples, someSampler) # anonymous
      n = length(varRanges)
      if n != $N error("Takes "*string($N)*" ranges, "*string(n)*" provided") end
      @hyperopt for i=1:numOfSamples, sampler=someSampler
        model([$(pars...)])
      end
    end
  end
  # don't know a way to interpolate to for header, so direct Expr mutation
  # bear in mind this bakes in the varRange values, not expressions.
  forassigns = funcexpr.args[end].args[end].args[end].args[end].args[begin].args
  append!(forassigns, ( :( $(pars[j]) = varRanges[$j] ) for j in 1:N))
  funcexpr
end

# could be const f for type stability
# to see the resulting expression, don't do the eval yet
f = eval(genFuncExpr(3))
f(LinRange.([0:0.01:1, 0:0.02:2, 0:0.03:6]), 10, Hyperopt.RandomSampler())

```

That seems to do what you intended, but again, not sure if `@hyperopt` can accommodate this, so you should check if `f` works.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [November 23, 2023, 8:01am UTC](https://discourse.julialang.org/t/how-to-generate-a-customized-for-loop-code/106552/7 "2023-11-23T08:01:17Z")

</div>

Something about this whole approach feels off to me, trying to do complicated metaprogramming to work around the limitations of some other complicated metaprogramming. I would take a step back and investigate what `@hyperopt` is doing and whether it contains building blocks that can be reused for the desired generalizations.