Meta-metaprogramming practical examples

General Usage
, ,
1

From Tom Kwongs’ excellent Hands-on Design Patterns and Best Practices with Julia:

It is possible to have a quoted expression that contains another quoted expression. […]

:( :( x = 1 ) ) |> dump

[…] It can be useful, however, if you need to generate code for
macros. I would definitely not suggest that you go more than two-levels deep and write
meta-meta-metaprograms!

I understand that this is a niche, probably with a few uses, and probably with even fewer justifiable uses. That being said, it piqued my interest — do you know of any use cases of nested quoted expressions, i.e. meta-metaprogramming, necessitated in a real-world scenario? What could this possibly be useful for?

2

There are some things that you cannot do without metaprogramming. One is creating new types of literals.

For example, consider my StaticStrings.jl package. Without macros, you would have to first create a normal String literal, pass it to the constructor, and then get a StaticString. With a string macro, you can create a StaticString literal without ever creating a String.

julia> using StaticStrings

julia> foo() = static"foo" # string macro
foo (generic function with 1 method)

julia> bar() = StaticString("foo") # constructor
bar (generic function with 1 method)

# They both return the same thing
julia> foo()
static"foo"3

julia> bar()
static"foo"3

julia> @code_lowered foo()
CodeInfo(
1 ─     return "foo"
)

julia> @code_lowered bar()
CodeInfo(
1 ─ %1 = Main.StaticString("foo")
└──      return %1
)

# The LLVM IR for foo is very simple. It just returns a static array of three bytes representing foo.
julia> @code_llvm foo()
;  @ REPL[22]:1 within `foo`
; Function Attrs: uwtable
define [1 x [3 x i8]] @julia_foo_847() #0 {
top:
  ret [1 x [3 x i8]] [[3 x i8] c"foo"]
}

# The LLVM IR for bar involves allocating dynamic memory for the String, initializing it, and then populating it
julia> @code_llvm bar()
;  @ REPL[23]:1 within `bar`
; Function Attrs: uwtable
define [1 x [3 x i8]] @julia_bar_849() #0 {
top:
  %gcframe4 = alloca [3 x {}*], align 16
  %gcframe4.sub = getelementptr inbounds [3 x {}*], [3 x {}*]* %gcframe4, i64 0, i64 0
  %0 = bitcast [3 x {}*]* %gcframe4 to i8*
  call void @llvm.memset.p0i8.i32(i8* noundef nonnull align 16 dereferenceable(24) %0, i8 0, i32 24, i1 false)
  %1 = getelementptr inbounds [3 x {}*], [3 x {}*]* %gcframe4, i64 0, i64 2
  %2 = bitcast {}** %1 to [1 x {}*]*
  %3 = call {}*** inttoptr (i64 140720661134992 to {}*** ()*)() #5
; ┌ @ C:\Users\kittisopikulm\.juliatest\packages\StaticStrings\QNj2T\src\convert.jl:31 within `StaticString`
   %4 = bitcast [3 x {}*]* %gcframe4 to i64*
   store i64 4, i64* %4, align 16
   %5 = getelementptr inbounds [3 x {}*], [3 x {}*]* %gcframe4, i64 0, i64 1
   %6 = bitcast {}** %5 to {}***
   %7 = load {}**, {}*** %3, align 8
   store {}** %7, {}*** %6, align 8
   %8 = bitcast {}*** %3 to {}***
   store {}** %gcframe4.sub, {}*** %8, align 8
   store {}* inttoptr (i64 1296121107536 to {}*), {}** %1, align 16
   %9 = call [3 x i8] @j_Tuple_851([1 x {}*]* nocapture readonly %2) #0
   %.fca.0.extract = extractvalue [3 x i8] %9, 0
   %.fca.1.extract = extractvalue [3 x i8] %9, 1
   %.fca.2.extract = extractvalue [3 x i8] %9, 2
; └
  %.fca.0.0.insert = insertvalue [1 x [3 x i8]] zeroinitializer, i8 %.fca.0.extract, 0, 0
  %.fca.0.1.insert = insertvalue [1 x [3 x i8]] %.fca.0.0.insert, i8 %.fca.1.extract, 0, 1
  %.fca.0.2.insert = insertvalue [1 x [3 x i8]] %.fca.0.1.insert, i8 %.fca.2.extract, 0, 2
  %10 = load {}*, {}** %5, align 8
  %11 = bitcast {}*** %3 to {}**
  store {}* %10, {}** %11, align 8
  ret [1 x [3 x i8]] %.fca.0.2.insert
}
1 Like
3

That’s a cool package and a nice way to use macros to your advantage. I skimmed through the source code and it seems to me this operates on a quoted expression, instead of a quoted expression that contains another quoted expression, i.e. it’s metaprogramming instead of meta-metaprogramming as per the original question. Am I missing something here?

4

I think the distinction between metaprogramming and meta-metaprogramming is not actually very useful to make. A macro that is able to process all valid Julia expressions by definition can handle arbitrarily deep nesting of quotation since Julia syntax already includes nesting.

5

We are dealing with literals, which are special in the metaprogramming space since quoting a literal is the literal itself.

julia> :(5)
5                                                                   

julia> :("hello")                 
"hello"

julia> eval(:(:("hi")))           
"hi"

Consider an alternative construction of the macros:

julia> macro foo_str(ex)                     
    ex2 = unescape_string(ex)                                           
    quote                                 
        StaticString($ex2)            
    end                           
end                        
@foo_str (macro with 1 method)                                      

julia> foo() = foo"hello"         
foo (generic function with 1 method)                                                                  

julia> @code_llvm foo()           
;  @ REPL[71]:1 within `foo`      
define [1 x [5 x i8]] @julia_foo_798() #0 {                     
top:                                
    %gcframe6 = alloca [3 x {}*], align 16                              
    %gcframe6.sub = getelementptr inbounds [3 x {}*], [3 x {}*]* %gcframe6, i64 0, i64 0                  
    %0 = bitcast [3 x {}*]* %gcframe6 to i8*                            
...

This still results in quite complicated code.

The notable thing about the string literal in StaticStrings.jl is that we receive an expression, a literal, manipulate that in the meta macro space, recieve a value, and then inject that value into a quoted expression as a literal.

Metaprogramming by itself would be turning the literal into an expression surrounding that literal. This is a meta-meta programming because I have done some actual computation in the meta space rather than just building a new expression to be evaluated, and then interpolated the result of that computation into a new expression as if it were a literal.

Thus, the result of the macro and subsequent compilation is not the dynamic allocation of a String and subsequent call to a constructor. Rather, it looks as if I had just returned a 64-bit number literal.

julia> g() = static"Accurate"
g (generic function with 1 method)

julia> @code_native g()
...
	movq	%rdi, %rax
	movabsq	$7310575239352771393, %rcx      # imm = 0x6574617275636341
	movq	%rcx, (%rdi)
	retq

In the above example, “Accurate” got converted into the number 7310575239352771393 or 0x6574617275636341 in hexadecimal.

julia> n = Int(0x6574617275636341)
7310575239352771393

julia> reinterpret(UInt8, [n]) .|> Char
8-element Vector{Char}:
 'A': ASCII/Unicode U+0041 (category Lu: Letter, uppercase)
 'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)
 'c': ASCII/Unicode U+0063 (category Ll: Letter, lowercase)
 'u': ASCII/Unicode U+0075 (category Ll: Letter, lowercase)
 'r': ASCII/Unicode U+0072 (category Ll: Letter, lowercase)
 'a': ASCII/Unicode U+0061 (category Ll: Letter, lowercase)
 't': ASCII/Unicode U+0074 (category Ll: Letter, lowercase)
 'e': ASCII/Unicode U+0065 (category Ll: Letter, lowercase)

That number happens to encode the eight ASCII bytes to spell “Accurate”.