# What exactly code lowering is an how to do "unlowering"?

**URL:** <https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315>\
**Category:** General Usage\
**Created:** [January 5, 2017, 9:32pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315 "2017-01-05T21:32:17Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [January 5, 2017, 9:32pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/1 "2017-01-05T21:32:17Z")

</div>

1. I’ve seen a number of snippets produced by `code_lowered` and have rough intuition of what it does, but what’s exact definition of code lowering? Does “lower” means “more low-level”?

2. In my package I need to get an expression of a function. I do it using the code at the end of this post, but essentially I get it using `Base.uncompressed_ast` which, as far as I understand, returns lowered code that is a bit hard to work with. Is there a way to “undo” lowering _or_ get function expression exactly as it has been written?

* * *

The code for getting a function’s expression:

```julia
    function replace_slots(ex::Expr, slotnames::Vector)
        new_args = Array(Any, length(ex.args))
        for (i, arg) in enumerate(ex.args)
            if isa(arg, Slot)
                new_args[i] = slotnames[arg.id]
            elseif isa(arg, Expr)
                new_args[i] = replace_slots(arg, slotnames)
            else
                new_args[i] = arg
            end
        end
        new_ex = Expr(ex.head, new_args...)
        return new_ex
    end

    """Extract arguments and (sanitized) expression of a function body"""
    function funexpr(f::Function, types::Vector{DataType})
        ms = methods(f, types).ms
        length(ms) != 1 && error("Found $(length(ms)) methods for function $f " *
                                 "with types $types, expected exactly 1 method")
        lambda = ms[1].lambda_template
        slot_ex_arr = Base.uncompressed_ast(lambda)
        slot_ex = sanitize(Expr(:block, slot_ex_arr...))
        slotnames = lambda.slotnames
        ex = replace_slots(slot_ex, slotnames)
        # 1st arg is a function name, next `lambda.nargs-1` are actual arg names                                                                                                     
        args = map(Symbol, slotnames[2:lambda.nargs])
        return args, sanitize(ex)
    end

```

Example function:

```julia
f(a, b) = sum(a' * b + a * b')

```

Output of `funexpr` / `Base.uncompressed_ast`:

```julia
Main.sum(Main.+(Main.Ac_mul_B(a,b),Main.A_mul_Bc(a,b)))

```

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [January 5, 2017, 10:40pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/2 "2017-01-05T22:40:43Z")

</div>

I have been working on Sugar.jl for various reasons, which does exactly this:  
[https://github.com/SimonDanisch/Sugar.jl](https://github.com/SimonDanisch/Sugar.jl)

Sorry for the sparse documentation, but I’m running out of time at the moment.  
I hope to finish and clean things up soon!

---

<div class="post-metadata">

**Author:** ![ihnorton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihnorton/32/26_2.png) [@ihnorton](https://discourse.julialang.org/u/ihnorton)\
**Post date:** [January 5, 2017, 11:49pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/3 "2017-01-05T23:49:36Z")

</div>

> [@dfdx](#):
>
> but what’s exact definition of code lowering? Does “lower” means “more low-level”?

“Lowering” means transforming code from Julia’s high-level syntax (`if`, `for`, `while`, etc.) to a smaller set of common primitives (calls, conditionals, labels, gotos). See discussion and definitions of [primitive node types and expressions](http://docs.julialang.org/en/stable/devdocs/ast) and [execution flow](http://docs.julialang.org/en/stable/devdocs/eval/#julia-execution) in the devdocs.

> [@dfdx](#):
>
> Is there a way to “undo” lowering

A Julia lowered-form decompiler _could_ be written; it’s a [sub-subfield](https://en.wikipedia.org/wiki/Decompiler) of computer science. But I don’t think one currently exists. There’s probably a limit to how closely the returned expression can match the original.

> [@dfdx](#):
>
> get function expression exactly as it has been written?

[Macros](http://docs.julialang.org/en/stable/manual/metaprogramming/)? They exist to capture and transform surface-level syntax.

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [January 6, 2017, 12:27pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/4 "2017-01-06T12:27:13Z")

</div>

Looks interesting, thanks! Is there anything to get not lowered version of function expression?

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [January 6, 2017, 12:33pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/5 "2017-01-06T12:33:10Z")

</div>

> [@ihnorton](#):
>
> “Lowering” means transforming code from Julia’s high-level syntax (if, for, while, etc.) to a smaller set of common primitives (calls, conditionals, labels, gotos). See discussion and definitions of primitive node types and expressions and execution flow in the devdocs.

Thanks! I will take a look at it.

> [@ihnorton](#):
>
> Macros? They exist to capture and transform surface-level syntax.

In my case it’s not an option, unfortunately. To give you some context: I’m working on a library for source-to-source symbolic differentiation. Even I force users to wrap the function to differentiate into a macro, this function internally may call many other functions that user doesn’t have access to. Currently with `funexpr` as above I’m able to parse about 85% of functions, but lowering seems to break my code for the other 15%.

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [January 6, 2017, 12:53pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/6 "2017-01-06T12:53:43Z")

</div>

With e.g.: `Sugar.sugared(func, (Types...), code_typed)`  
`code_typed` indicated from which lowered version to start and in this case will result in typed expressions.  
If the code is not high enough, you need to insert additional pattern match passes.  
Like in:  
[https://github.com/SimonDanisch/Sugar.jl/blob/master/src/ast\_tools.jl#L194](https://github.com/SimonDanisch/Sugar.jl/blob/master/src/ast_tools.jl#L194)  
[https://github.com/SimonDanisch/Sugar.jl/blob/master/src/sugarcoating.jl#L20](https://github.com/SimonDanisch/Sugar.jl/blob/master/src/sugarcoating.jl#L20)

Sorry if there is quite a bit of noise, unneeded functions and no comments 😉  
I just very recently got this to a point where it works for my use case and still haven’t cleaned up.

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [January 6, 2017, 12:58pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/7 "2017-01-06T12:58:54Z")

</div>

There is also: `macro_form(f, types)`  
Which tries to look up the source in the file and then returns the parsed expression exactly like a macro would.  
This will fails for clojures, which e.g. broadcast produces and REPL code, so I kinda stopped working on that function.

[https://github.com/SimonDanisch/Sugar.jl/blob/master/src/lambdas.jl#L209](https://github.com/SimonDanisch/Sugar.jl/blob/master/src/lambdas.jl#L209)

---

<div class="post-metadata">

**Author:** ![jameson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jameson/32/23_2.png) [@jameson](https://discourse.julialang.org/u/jameson)\
**Post date:** [January 6, 2017, 5:47pm UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/8 "2017-01-06T17:47:59Z")

</div>

> [@dfdx](#):
>
> To give you some context: I’m working on a library for source-to-source symbolic differentiation.

While the exact contents of lowered expressions is not as stable as the front-end parsing, it may be better suited for you in doing this sort of transform. You can sort of think of it as a normalization pass that converts the parse syntax and annotates it with lots of crucial information (like the targets of loops and other branch, the scope of variables, etc)

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [January 8, 2017, 1:02am UTC](https://discourse.julialang.org/t/what-exactly-code-lowering-is-an-how-to-do-unlowering/1315/9 "2017-01-08T01:02:16Z")

</div>

After playing around a bit with Sugar.jl I think `macro_form` is good enough for most practical uses, so I think in the next iteration of that part I’ll use it with fallback to lowered code. Thanks!

As for using lowered expressions instead of high-level ones, although it’s a reasonable approach, I’d like to avoid it when possible to keep parsed and generated expressions human-readable. And by “human-readable” I mean not only by humans who know Julia AST details 🙂