Metaprogramming design patterns

General Usage
,
1

Feeling in a refactoring mood I decided to bravely tackle some performance bottlenecks that were present in Genie’s codebase. I decided to go for the low hanging fruits and squash The Evil eval.

A quick search revealed 3 big (anti) patterns involving eval. Some of them are pretty old in my codebase and I googled them many times, without being able to find a definitive (or even recent) solution.

#1. dynamically invoking functions.
Once upon a time there was invoke. It got kicked in v0.5 and no clear upgrade path was indicated (at least not for me when reading the “what’s new”). Hello eval!

Fortunately, this issue was greatly addressed by @StefanKarpinski in a StackOverflow reply - the way to do it is with getfield.

My only comment here is that invoking nested functions (like A.B.C.foo) gets horribly complex - and to be honest it took me a while to understand why Julia throws an error. The issue is, getfield has no support for nested fields - it would be awesome if it did, nesting 3-4 levels of getfield is horrible.

#2. dynamically generating functions (and optionally exporting them)
This is a common design pattern when developing DSLs. For example, Genie provides a templating system which allows writing views in pure Julia while still looking like an HTML structure (think Ruby’s HAML).

It looks something like:

body(:class => "foo", :onload => "...") do 
  h1() do 
    "Welcome!"
  end
  p() do 
     # more stuff here
  end
end

All these functions are simple utility methods that delegate to a function like:

function elem(content::Function, html_tag::Symbol)

And I have an array of html tags, like:

[:body, :h1, :p]

What is the right way of generating (and optionally exporting) the utility functions, based on the array, without using eval?

#3. dynamically include and using modules
I’m using these a lot in a factory style of design pattern. Basically I have various modules that rely on “adapters”. For example for caching or for sessions. Both cache and sessions can be stored either on the file system, a RDBMS or a NoSQL backend. The adapters expose a common interface and the framework just delegates to the underlying adapter for persisting and retrieving the data.

The adapters are user configurable in a settings file - but as the configuration is loaded very early in the app’s life cycle, the actual modules are not available yet. So the config uses Symbols to reference the adapter name - which later (when/if needed) has to be actually included and referenced.

What’s the right way of doing it, without using eval?

===

Sorry for the long write and thanks in advance for your time – if you have other design patterns please add them, I think that having a reference for this in one place will be very useful. I’m happy to aggregate it and turn it into a page we can put into the docs or on Julia’s blog. I think many users make the mistake of reaching for evil eval, especially when coming from interpreted languages.

===

Btw, here is Genie’s repo, if you feel so inclined to take a look at the code and help track performance issues I’ll be forever grateful, I’m very good at taking constructive criticism :smile: GitHub - GenieFramework/Genie.jl: 🧞The highly productive Julia web framework

2 Likes
2

What do you mean “got kicked”?? It’s still in 0.6 and is even faster now. And what do you mean by “dynamically invoking” anyway…

Some of the things you mentioned seems very dynamic so you’ll never be able to do that in a local scope without eval. Of course if you can live with less flexibility generated function might be able to do some of these. Using eval for a one time setup shouldn’t be an anti-pattern. You just want to generate the necessary code once and don’t use eval after that.

3

Oh?! My understanding was that the function was deprecated in 0.5 for being slow? I googled it quickly but I can’t find exactly where I’ve read. I must’ve gotten it wrong then, my apologies (I will edit my question to reflect this).

By “dynamically invoking” I mean invoking certain methods without knowing in advance which they are. Both the methods and the calls are defined and triggered by the user.

The classical example is a web app router in an MVC framework.
A. the developer (user of the framework) defines his methods within controllers – in our case the controllers being also developer (framework user) created modules
B. also the developer defines mappings using URI and Module.function pairs
C. the end user (users of the web app) invokes the methods by means of URIs - via predefined routes.

Ex:

# A
module Foos
function bar()
# stuff
end
end

# B
route("/foos/bar", "Foos#bar")

In effect, a request to http://yourdomain/foos/bar would invoke Foos.bar()

Ok, so there can be use cases when eval simply is the best way to approach a problem - indeed, it is a one time operation upon loading the module for the first time.

4

You don’t need/can’t use invoke for this. You just need to getfield that function and call it.

5

Thanks, that’s what I’m doing.

For the other cases, is it OK to go with eval?

Thanks

6

Why not just store the function object itself?

routes["/foo/bar"] = Foo.bar
routes["/foo/bar"]()
7

Hmmm, interesting idea!

Possible issues:
a. the router is loaded early on when the controllers are not yet available - although this can be changed; but,
b. won’t this be a performance issue? Each request is handled in a Task by HttpServer. The way it is right now, only the corresponding controller is brought into scope. If I bring all the controller modules into scope in order to reference their functions in the router, multiplied for thousands of requests, it seems to me that it will require considerable more RAM and won’t scale, especially for large codebases…

8

Scrap that, using directly the function is a good idea - the controllers represent only a small part of the code base so I doubt they’d have a significant impact in most cases.