# Skipping LineNumberNodes in v0.7 macros

**URL:** <https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547>\
**Category:** General Usage\
**Tags:** metaprogramming\
**Created:** [June 8, 2018, 7:00pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547 "2018-06-08T19:00:26Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 8, 2018, 7:00pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/1 "2018-06-08T19:00:26Z")

</div>

```julia
ex = :(begin
    p1, (X,Y,Z) --> 0
    hill(X,p2,100.,-4), 0 --> Y
    hill(Y,p3,100.,-4), 0 --> Z
    hill(Z,p4,100.,-4), 0 --> X
    hill(X,p5,100.,6), 0 --> R
    hill(Y,p6,100.,4)*0.002, R --> 0
    p7, 0 --> S
    R*p8, S --> SP
    p9, SP + SP --> SP2
    p10, SP2 --> 0
end)

julia> ex.args
20-element Array{Any,1}:
 :(#= REPL[5]:2 =#)
 :((p1, $(Expr(:-->, :((X, Y, Z)), 0))))
 :(#= REPL[5]:3 =#)
 :((hill(X, p2, 100.0, -4), $(Expr(:-->, 0, :Y))))
 :(#= REPL[5]:4 =#)
 :((hill(Y, p3, 100.0, -4), $(Expr(:-->, 0, :Z))))
 :(#= REPL[5]:5 =#)
 :((hill(Z, p4, 100.0, -4), $(Expr(:-->, 0, :X))))
 :(#= REPL[5]:6 =#)
 ⋮
 :((hill(Y, p6, 100.0, 4) * 0.002, $(Expr(:-->, :R, 0))))
 :(#= REPL[5]:8 =#)
 :((p7, $(Expr(:-->, 0, :S))))
 :(#= REPL[5]:9 =#)
 :((R * p8, $(Expr(:-->, :S, :SP))))
 :(#= REPL[5]:10 =#)
 :((p9, $(Expr(:-->, :(SP + SP), :SP2))))
 :(#= REPL[5]:11 =#)
 :((p10, $(Expr(:-->, :SP2, 0))))

julia> ex.args[2]
:((p1, $(Expr(:-->, :((X, Y, Z)), 0))))

julia> ex.args[1]
:(#= REPL[5]:2 =#)

julia> typeof(ex.args[1])
LineNumberNode

```

In v0.6 I used to `for a in ex.args` but now that gives these extra LineNumberNodes in there. Is there a canonical way to remove them or skip them when iterating? Are they always the odd values?

---

<div class="post-metadata">

**Author:** ![NaOH](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/naoh/32/5632_2.png) [@NaOH](https://discourse.julialang.org/u/NaOH)\
**Post date:** [June 8, 2018, 7:21pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/2 "2018-06-08T19:21:07Z")

</div>

> [@ChrisRackauckas](#):
>
> ex = :(begin p1, (X,Y,Z) –\> 0 hill(X,p2,100.,-4), 0 –\> Y hill(Y,p3,100.,-4), 0 –\> Z hill(Z,p4,100.,-4), 0 –\> X hill(X,p5,100.,6), 0 –\> R hill(Y,p6,100.,4)_0.002, R –\> 0 p7, 0 –\> S R_p8, S –\> SP p9, SP + SP –\> SP2 p10, SP2 –\> 0 end)

You can use `MacroTools.prettify`:

```julia
julia> using MacroTools: prettify

julia> ex = quote
           p1, (X,Y,Z) --> 0
           hill(X,p2,100.,-4), 0 --> Y
           hill(Y,p3,100.,-4), 0 --> Z
           hill(Z,p4,100.,-4), 0 --> X
           hill(X,p5,100.,6), 0 --> R
           hill(Y,p6,100.,4)*0.002, R --> 0
           p7, 0 --> S
           R*p8, S --> SP
           p9, SP + SP --> SP2
           p10, SP2 --> 0
       end |> prettify
quote
    (p1, $(Expr(:-->, :((X, Y, Z)), 0)))
    (hill(X, p2, 100.0, -4), $(Expr(:-->, 0, :Y)))
    (hill(Y, p3, 100.0, -4), $(Expr(:-->, 0, :Z)))
    (hill(Z, p4, 100.0, -4), $(Expr(:-->, 0, :X)))
    (hill(X, p5, 100.0, 6), $(Expr(:-->, 0, :R)))
    (hill(Y, p6, 100.0, 4) * 0.002, $(Expr(:-->, :R, 0)))
    (p7, $(Expr(:-->, 0, :S)))
    (R * p8, $(Expr(:-->, :S, :SP)))
    (p9, $(Expr(:-->, :(SP + SP), :SP2)))
    (p10, $(Expr(:-->, :SP2, 0)))
end

julia>

```

You also don’t need `:(begin; ...; end)`, for multi-line expressions you could simply use `quote; ...; end`.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 8, 2018, 7:36pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/3 "2018-06-08T19:36:21Z")

</div>

You can also use `Reduce.linefilter`, which removes all comments like that from a piece of code.

```nohighlight
julia> Reduce.linefilter(ex)
quote 
    (p1, $(Expr(:-->, :((X, Y, Z)), 0)))
    (hill(X, p2, 100.0, -4), $(Expr(:-->, 0, :Y)))
    (hill(Y, p3, 100.0, -4), $(Expr(:-->, 0, :Z)))
    (hill(Z, p4, 100.0, -4), $(Expr(:-->, 0, :X)))
    (hill(X, p5, 100.0, 6), $(Expr(:-->, 0, :R)))
    (hill(Y, p6, 100.0, 4) * 0.002, $(Expr(:-->, :R, 0)))
    (p7, $(Expr(:-->, 0, :S)))
    (R * p8, $(Expr(:-->, :S, :SP)))
    (p9, $(Expr(:-->, :(SP + SP), :SP2)))
    (p10, $(Expr(:-->, :SP2, 0)))
end

```

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [June 8, 2018, 8:23pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/4 "2018-06-08T20:23:27Z")

</div>

But presumably one should keep them for better back-traces, right? So maybe

```julia
out = []
for a in ex.args
  if a isa LineNumberNode
    push!(out, a)
  else
    # do something
    push!(out, somthing)
  end
end
...

```

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 8, 2018, 9:22pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/5 "2018-06-08T21:22:44Z")

</div>

A `for` loop won’t suffice, you need to go recursively into the ASTree to access all possible comments. This is why `Reduce.linefilter` is recursively defined with `while` loops

[https://github.com/chakravala/Reduce.jl/blob/master/src/parser.jl#L423-L443](https://github.com/chakravala/Reduce.jl/blob/master/src/parser.jl#L423-L443)

This is backwards compatible with 0.6 and 0.7 simultaneously.

---

<div class="post-metadata">

**Author:** ![MikeInnes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikeinnes/32/3656_2.png) [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)\
**Post date:** [June 8, 2018, 9:37pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/6 "2018-06-08T21:37:29Z")

</div>

Note that `prettify` does multiple things and is mainly aimed at showing code in the REPL. For macros you probably want `striplines`.

`prewalk` [makes the recursion a lot nicer here](https://github.com/MikeInnes/MacroTools.jl/blob/56d50309418f20d6a48a7b75c620c7ee53a30f29/src/utils.jl#L65) 🙂

---

<div class="post-metadata">

**Author:** ![NaOH](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/naoh/32/5632_2.png) [@NaOH](https://discourse.julialang.org/u/NaOH)\
**Post date:** [June 8, 2018, 9:39pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/7 "2018-06-08T21:39:21Z")

</div>

`MacroTools.prettify` does more than just removing line nodes, so just take that into account:

[https://github.com/MikeInnes/MacroTools.jl/blob/master/src/utils.jl#L387-L393](https://github.com/MikeInnes/MacroTools.jl/blob/master/src/utils.jl#L387-L393)

There is also the `@q` macro:

[https://github.com/MikeInnes/MacroTools.jl/blob/master/src/utils.jl#L18-L24](https://github.com/MikeInnes/MacroTools.jl/blob/master/src/utils.jl#L18-L24)

@mauro3 macro tools also have `prewalk` and `postwalk` that are used to traverse expressions recursively instead of iterating on them.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 8, 2018, 9:48pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/8 "2018-06-08T21:48:44Z")

</div>

However, it turns out that my implementation in `Reduce` is significantly faster than `striplines` from `MacroTools`, as the benchmark shows

```nohighlight
julia> @btime Reduce.linefilter(ex)
  21.442 μs (0 allocations: 0 bytes)
quote 
    (p1, $(Expr(:-->, :((X, Y, Z)), 0)))
    (hill(X, p2, 100.0, -4), $(Expr(:-->, 0, :Y)))
    (hill(Y, p3, 100.0, -4), $(Expr(:-->, 0, :Z)))
    (hill(Z, p4, 100.0, -4), $(Expr(:-->, 0, :X)))
    (hill(X, p5, 100.0, 6), $(Expr(:-->, 0, :R)))
    (hill(Y, p6, 100.0, 4) * 0.002, $(Expr(:-->, :R, 0)))
    (p7, $(Expr(:-->, 0, :S)))
    (R * p8, $(Expr(:-->, :S, :SP)))
    (p9, $(Expr(:-->, :(SP + SP), :SP2)))
    (p10, $(Expr(:-->, :SP2, 0)))
end

julia> @btime MacroTools.striplines(ex)
  427.192 μs (443 allocations: 20.70 KiB)
quote 
    (p1, $(Expr(:-->, :((X, Y, Z)), 0)))
    (hill(X, p2, 100.0, -4), $(Expr(:-->, 0, :Y)))
    (hill(Y, p3, 100.0, -4), $(Expr(:-->, 0, :Z)))
    (hill(Z, p4, 100.0, -4), $(Expr(:-->, 0, :X)))
    (hill(X, p5, 100.0, 6), $(Expr(:-->, 0, :R)))
    (hill(Y, p6, 100.0, 4) * 0.002, $(Expr(:-->, :R, 0)))
    (p7, $(Expr(:-->, 0, :S)))
    (R * p8, $(Expr(:-->, :S, :SP)))
    (p9, $(Expr(:-->, :(SP + SP), :SP2)))
    (p10, $(Expr(:-->, :SP2, 0)))
end

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 9, 2018, 1:17pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/9 "2018-06-09T13:17:26Z")

</div>

`prettify` broke something on v0.6, Reduce is far too large of a dependency to even consider here, and `striplines` worked well with no hassle, so `striplines` it is.

[https://github.com/JuliaDiffEq/DiffEqBiological.jl/commit/792e7b98cf0dc9389765c597c040b77fcdb6d658](https://github.com/JuliaDiffEq/DiffEqBiological.jl/commit/792e7b98cf0dc9389765c597c040b77fcdb6d658)

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 9, 2018, 3:52pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/10 "2018-06-09T15:52:04Z")

</div>

created a new package called `SyntaxTree` that now contains `linefilter` method

> **[GitHub - chakravala/SyntaxTree.jl: Toolset for modifying Julia AST and...](https://github.com/chakravala/SyntaxTree.jl)**
>
> Toolset for modifying Julia AST and characteristic values - GitHub - chakravala/SyntaxTree.jl: Toolset for modifying Julia AST and characteristic values

---

<div class="post-metadata">

**Author:** ![MikeInnes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikeinnes/32/3656_2.png) [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)\
**Post date:** [June 11, 2018, 12:16pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/11 "2018-06-11T12:16:50Z")

</div>

I would generally discourage using mutation with `Expr`s, given the safety you get (and performance usually matters less when from macro code that runs once).

But you could no doubt implement an in-place `prewalk!` for MacroTools and get the best of both versions 🙂

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 11, 2018, 1:08pm UTC](https://discourse.julialang.org/t/skipping-linenumbernodes-in-v0-7-macros/11547/12 "2018-06-11T13:08:51Z")

</div>

In my case, I needed something with high performance, since I need to apply this method many times, it would take up a significant chunk of processing time if it was too slow, so I opted for the fastest way possible by making a single purposed method for this task. I just simply don’t need a generalized walk method for this, I really needed a special purpose method that does it as fast as possible, that’s why I made it.

So in my special situations, the performance does make a big difference for something like this.

I think you could also write a code generator that writes special purpose code for parsing through syntax trees based on what operation it is, such as the `linefilter` method, then you can also gain performance.