Metaprogramming/code generation inside a function

General Usage
1

Hi,

First I’ll quickly sketch the goal:
I’ve been trying to read an input file which has a couple of blocks (control,system,electrons,ions,cell). Each block starts with a & and ends with a /. Inside each block there are several keywords and values. For each of these blocks I want to create a dictionary and store the key-value pairs.

Now, since the loop inside each block is exactly the same, I thought this would a perfect opportunity to get some metaprogramming/code generation in there.

I’ve been experimenting with stuff to try and get this to work, but I can’t seem to figure out just how exactly do it. I’ve looked at the documentation and other resources on this, but I don’t fully get how code inside function declarations gets parsed.

The code looks like this:

function read_QEInput(filename,T=Float32)
  blocks = [:control,:system,:electrons,:ions,:cell] 
  :($(block...) = [Dict{Symbol,Any}() for i=1:5])
  open(filename) do f
    while !eof(f)
      line = readline(f)
      for block in blocks
        if contains(line,"&$(Symbol(block))")
          line = readline(f)
          while !contains(line,"/")
            if contains(line,"!")
              line = readline(f)
              continue
            end
            split_line = filter(x->x!="",strip.(split(line,",")))
            for s in split_line
              key,val = String.(strip.(split(s,"=")))
              :($block[Symbol(key)] = isnull(tryparse(T,val)) ? val : get(tryparse(T,val)))
            end
            line = readline(f)
          end
        end
      end
    end
  end
  return control,system,electrons,ions,cell
end

What I’m trying to accomplish is first to initiate the dictionaries, basically to get to an expression like this:
control,system,electrons,ions,cell = [Dict{Symbol,Any} for i=1:5] ,
then to basically generate an if contains(line,"&block") section for each of the blocks that I define.

However none of my tries amounted to anything that actually worked. So I guess my questions are: am I trying to do something that is impossible? Am I approaching it from the wrong angle?

I know I could just define another function or maybe a macro which handles this block of code, but I think it would be more clear like this + it might be instructive to learn about the workings of the language.

Thanks!

2

Not sure if that’s what you want or if it’s the recommended way to do it, but have you tried looking into @eval?

(also, if you are writing a parser for Quantum Espresso input files, please put the code online somewhere! :-))

3

Don’t do that…

No, what you have there seems like normal loop, you don’t need and shouldn’t use any meta-programming “inside the function” here.

1 Like
4

Yes, I thought that as well. But then could you elaborate on how to assign Dicts to each of the variables (the ones in blocks are supposed to be the variables)?
This obviously doesn’t work:

blocks... = [Dict{Symbol,Any}() for i=1:5]

neither do variations on that construct.

Also if i recall correctly I get an error in setindex!(:Symbol,:Symbol,:String/:Any) in the body of the for block in blocks loop.

5

I tried that, putting the blocks=[…] outside the function and then @eval. But that results in a block is undefined issue for some reason.

6

You can’t. If the variable names are fixed, just write them out, if they are not fixed, then just use a dict to store them instead. i.e. Dict{Symbol,Dict{Symbol,Any}} (Also you can just use string since that’s what you get from parsing).

And similarly, this is impossible by design.

7

This is what I expected as well. So what I am trying to do inside of the block in blocks loop is impossible (interpreting the Symbol as a variable rather than the symbol)?

8

I’d go with one of your original options of defining another function to do the core part. Something like the following:

function read_QEInput(filename,T=Float32)
  open(filename) do f
    return (read_block(f, "control", T),
            read_block(f, "system", T),
            read_block(f, "electrons", T),
            read_block(f, "ions", T),
            read_block(f, "cell", T))
  end
end
9

Ok, thanks! So with regards to the uses of metaprogramming in these situations to generate code. Is it correct for me to think about it as to only use it to generate code that is evaluated in global scope, but not really for copy pasting code inside function bodies? For example, i’ve used it to define multiple functions that had the same general structure but slightly altered parts in the body, this worked well with the @eval macro + pushing the code into quote end blocks which then got pasted later.

10

Correct.

No. You can use a macro for that, as long as the information you need to generate the code is available from just syntactic information (at compile time). Generating code is a compile time operation so it is by definition impossible to do with only runtime features.

11

Ok, seems logical. Thanks for the clarification!